Distributed physics based training system and methods

ABSTRACT

A distributed simulation system is composed of simulator stations linked over a network that each renders real-time video imagery for its user from scene data stored in its data storage. The simulation stations are each connected with a physics farm that manages the virtual objects in the shared virtual environment based on their physical attribute data using physics engines, including an engine at each simulation station. The physics engines of the physics farm are assigned virtual objects so as to reduce the effects of latency, to ensure fair fight requirements of the system, and, where the simulation is of a vehicle, to accurately model the ownship of the user at the station. A synchronizer system is also provided that allows for action of simulated entities relying on localized closed loop controls to cause the entities to meet specific goal points at specified system time points.

RELATED APPLICATIONS

This application claims the priority of U.S. provisional applicationSer. No. 60/740,579 filed Nov. 28, 2005, which is herein incorporated byreference in its entirety.

FIELD OF THE INVENTION

This invention relates to the field of simulators, and more particularlyto the field of simulator systems for multiple users for training inmilitary exercises or military vehicle operations, especially suchsystems for users that are geographically distributed relative to eachother and connected by communications links such as a network or theInternet.

BACKGROUND OF THE INVENTION

It is known in the prior art to provide simulation systems for usersdistributed geographically relative to each other, potentially thousandsof miles apart, with communications between the various users beingtransmitted over a network, such as the Internet. In these systems, eachuser normally has a simulation station where the user can see on ascreen, a head-mounted display, or other display device, that user'sview of a shared virtual environment based on his virtual position init. The displayed scene is video prepared by an image generator at thesimulation station that renders serial frames of video using changingscene data that defines the visible objects in the simulated environmentin real-time. The user also can interact with and affect the virtualenvironment, as with simulated cockpit controls in an aircraftsimulator, simulated weapons systems, or other interface mechanismsknown in the art, and his interactions that change the scenery can beseen by him and all the other users.

In the armed forces, Distributed Interactive Simulations (DIS) arerecognized as being essential for training, system acquisition and testand evaluation. DIS may also be employed in non-military simulationapplications such as, e.g., simulations for air traffic control,airplane crash, traffic, land use, environmental, natural disaster,videogames (networked multiplayer video games) and the like. Within theU.S. Department of Defense, key components of DIS include high-fidelitycomputer-simulated individual entities (tanks, trucks, aircraft, etc.),interactions among entities hosted on different computers throughnetwork messages, and support for Human In Loop (HIL) interactions.

Using DIS, it is possible to create large-scale virtual representationsof real operational environments that are inexpensive enough to be usedrepeatedly. Accordingly, a set of DIS protocols were promulgated by theIEEE in 1995 and became the basis for much of the current generation ofnetworked simulations.

Modular Semi-Automated Forces (“ModSAF”) is an example of theimplementation and use of the DIS protocol in the armed forces forcost-effective training. ModSAF generally runs using a collection ofworkstations communicating over a network, typically a LAN. Eachworkstation is responsible for simulating a predetermined number ofentities, or simulated objects. These computer-generated Semi-AutomatedForces (“SAF”) are intended to mimic realistically the behaviors ofopposing forces or support forces within an exercise. The entity andenvironment models employed are typically quite detailed and theindividual simulators can interact by exchanging data using a messagingprotocol, the basic unit of which is referred to as a Protocol DataUnits (PDUs). PDUs are used in ModSAF to describe the state ofindividual entities, weapons firing, detonations, environmentalphenomena, command and control orders, etc.

In standard ModSAF, the PDUs are sent as UDP datagrams, which is anunreliable message-delivery mechanism, and therefore each entity statePDU typically contains a complete summary of the vehicle's currentstate. PDUs are transmitted/retransmitted at frequent, regular“heartbeat” intervals to compensate for dropped data packets. In usingsuch a system, assurance that each simulator receives and responds toall PDUs from all other simulators is extremely difficult, if notimpossible, as the number of simulators and simulated entities increasesdue to the avalanche of communication/network traffic that results fromdynamically moving entities continually updating their state.Human-In-Loop (HIL) entities tend to complicate matters even furthergiven their interactive and inherently unpredictable behavior.

A primary problem encountered in such distributed systems is that oflatency, meaning communications delay between the transmission from aremote site of data affecting the scene and the reception and display ofthat data at a user's station. Latency depends on distance and thequality of the communications links, and can commonly be 200milliseconds (a fifth of a second) or more. The consequence of latencyin multi-user situations is that the actions of a remote user do notaffect the scene viewed by the local user immediately, and this canresult in inconsistencies in the views seen by the users in thedistributed system.

To try to reduce some of the effects of latency, prior art systems haveused a kinematics approach in which, when information relating toposition and movement of a virtual object is received, it is checked fora time-stamp indicative of when precisely in system-time the data wasproduced. The movement of the affected object is then simply linearlyprojected or extrapolated to an estimate of its current position, basedon its position and velocity defined by the most recent data available,together with a Δt of the time-stamp from the current time, indicatingthe age of the data in system-time terms.

Linear projections of this type do not accurately extrapolate the realposition of objects under a number of circumstances, and in addition tothe positional errors associated with a pure linear projection of thelocation of the object, it is also possible that behavior of the objectwill be unrealistic if it is projected to a point that it could notreach in reality, such as, for example, when the position of a movingperson or vehicle is extrapolated to the other side of a solid wall,making it appear that the person or vehicle actually went through thewall.

Furthermore, especially in military simulation-based training,discrepancies between various users' views of the simulated world cancompromise principles of training, notably “fair fight” requirements.“Fair Fight” is a term that means no combatant has an unfair advantageover another due to differences in each participant's representation ofthe world. Differences in the viewed scenery of the participants canafford such an advantage to one user or another.

In addition, simulators for vehicles, especially aircraft, are usuallymodeled to run scripts when certain events occur to the user's ownship,such as when damage is done to the vehicle in combat. As an example, ina helicopter simulation, if a blade of the helicopter is shot at and itbreaks off, the simulator normally initiates a damaged-vehicle scriptthat is usually limited to a single type of damage scenario, and doesnot provide the realism of allowing a wide variety of effects for thevarious types of damage that may be done to the aircraft. Theselimitations reduce realism and also lessen to a degree the trainingvalue of the simulation.

SUMMARY OF THE INVENTION

It is accordingly an object of the invention to provide systems andmethods that avoids the drawbacks of the prior art.

In one aspect of the invention, this is achieved by a simulation stationfor use in a simulation system having a plurality of users eachassociated with a respective simulation station. The simulation stationcomprises a link to a communication network to which all simulationstations are linked, and computer accessible data storage storing scenedata. The scene data includes object data defining a plurality ofvirtual objects in a virtual environment. The object data includes foreach of the objects location data defining a location of the object inthe virtual environment and data defining characteristics of the virtualobject affecting its appearance. An image generator periodically rendersfrom the scene data a frame of video imagery corresponding to a view ofthe virtual environment from a position associated with the user of thesimulation station so that the user is provided with a real time view ofthe virtual environment. A display receives the video imagery anddisplays the imagery so as to be viewable by the associated user. Aphysics engine manages the virtual objects in the scene data bydetermining physics data for a subset of the virtual objects in thescene based on data defining physical attributes of the objects and thelocations thereof, and receiving, via the link, physics data regardingother of the virtual objects in the scene data. The physics engineinfluences the scene data so that the location data of the virtualobjects as defined in the scene data complies with physical rulesgoverning, based at least in part on the physical attribute data, themovement of the object in the virtual environment.

According to another aspect of the invention, a system is provided forgiving a shared simulated environment to a plurality of users eachhaving a respective user location in said simulated environment. Thesystem comprises a plurality of simulator stations each associated witha respective user and linked over a network so as to communicate witheach other. The simulator stations each include a data storage systemstoring scene data thereon and a computerized image generator generatingreal-time video as a stream of images each rendered based on the scenedata corresponding to a respective point in time. The video is displayedto one of the users associated with the simulator station using adisplay communicating with the image generator. The scene data includesobject data defining a position and physical attributes of virtualobjects in the shared simulated environment. A physics farm is connectedwith the simulator stations, and it interfaces with the scene data ofeach of the simulator stations and controls position data of virtualobjects of the scene data based on the defined physical attributes ofthe virtual objects and on physical rules that cause the virtual objectsto emulate real movement of objects having corresponding real physicalattributes. The physics farm comprises a plurality of physics enginesoperatively connected with the network so as to communicate with eachother, each having at least one of the physics engines at its location,and a physics manager assigning each of the virtual objects in theshared simulated environment to one of the physics engines. Each of thephysics engines determines if any of the virtual objects assignedthereto are in a physical conflict with any other objects in thesimulated environment. Responsive to such a determination, each physicsengine attempts to resolve the conflict of the objects locally, withoutreference to the other physics engines. If not resolved locally by thephysics engine, the conflict outcome for the virtual objects in theconflict is determined by the physics engines assigned the virtualobjects involved, and the conflict outcome is communicated bytransmitting a packet of data identifying the virtual object involvedand including conflict data corresponding to the conflict over thenetwork, from which packet of data the position or positions of thevirtual objects involved after the conflict can be determined.

According to a further aspect of the invention a method for providing aninteractive simulation of an environment to a plurality of userscomprises providing each of the users with a respective simulatorstation having a display displaying to the associated user videorendered by a computerized image generator from scene data stored in acomputer accessible data storage system. An input signal indicative ofan interactive command from the user for influencing the simulatedenvironment is received from one of the users at the respectivesimulator station. Data from the input signal defining the interactivecommand is derived as a force applied to one or more virtual objects inthe simulated environment. Physics processing of the force or torque onthe object or objects is performed using a distributed physics farmcomprising a plurality of physics engines. At least one of the physicsengines is operatively associated with the respective simulator station.Physics notification data on the object and a system time thereof ispublished to all of the physics engines in the physics farm. The physicsnotification data is received at a second of the physics enginesoperatively associated with another of the simulation stations. Physicsprocessing of the notification data of the object is performed at thesecond of the physics engines. The second physics engine computes aposition of the object based on the physics notification data and adifference between the system time thereof and a current system time.The second physics engine accesses locally data defining the positionsof other objects in the simulated environment, and the determination ofthe computed position includes a conflict check that incorporates intothe computation any conflict with other objects in the simulatedenvironment. The scene data at the associated simulation station ismodified such that the scene data reflects an updated position of theobject conforming to the computed position. The scene data is renderedto produce imagery showing the object at the computed position, and theimagery is displayed to the user of the second simulation station.

According to another aspect of the invention, a simulation systemcomprises a physics farm communications layer comprising acommunications backbone and a physics farm layer. The physics farm layerhas a plurality of distributed physics engines each communicativelylinked to the physics farm communications layer so as to transmit datato the other physics engines and to receive data therefrom. Each physicsengine stores physical attribute and location data for all of objects ina virtual environment. A physics farm management system selects thephysics engine that performs physics processing for each object in thevirtual environment, and a physics-based simulation API communicateswith the physics farm management system to allow access thereto byapplications. A physics based simulation application layer comprising atleast two distributed simulation stations provides interactivesimulation to respective users, including displaying to the respectiveuser imagery rendered from scene data stored at the simulation system.The scene data is modified by the physics farm layer to reflectinteraction of the users with the objects in the virtual environment.

According to still a further aspect of the invention, a method ofproviding distributed simulation of a plurality of vehicles comprisesproviding a plurality of simulation stations each having simulatedcontrols for a respective vehicle being simulated, a display, a storagemedium storing scene data defining positions of virtual objects in avirtual environment, and a physics engine. The physics engine storesphysics data for objects in the virtual environment, and the objectsinclude an ownship set of objects configured to correspond to thevehicles being simulated. Physics processing of the ownship set ofobjects for the simulation station is performed using the physics engineof each simulation station, and the resulting physics data istransmitted to the other simulation station or stations. The physicsdata is received at the other simulation station or stations and thescene data thereof is modified based on the physics data so that theother user or users can see the other station's ownship if appropriateto said user's point of view. The physics data for each ownship is usedto control the simulated behavior of the ownship of said simulationstation.

It is also an object of the invention to provide a simulation systemcomprising a plurality of computerized simulation stations connectedwith each other via a network. Each simulation station has a respectivecomputer-accessible data storage storing a respective version of scenedata corresponding to a virtual environment, a respective computerizedimage generator generating serial frames, each of which is a renderedview for a respective point of view in the virtual environment definedby the respective version of the scene data stored, and a display devicedisplaying said frames. The system further comprises a simulationcomputer system determining behavior of a virtual client entity in thevirtual environment and transmitting client movement data via thenetwork to the simulation stations. This client movement datacorresponds to the determined behavior, and it defines at least onelocation in the virtual environment to which the client entity is tomove, as well as time-constraint data indicative of a point in time bywhich the entity is to move to the location. Each simulation station hasa synchronizer that receives the client movement data and, basedthereon, causes the virtual object of the client entity in the scenedata of the simulation station to be modified so as to reflect movementof the client entity to the location by the point in time by which theentity is to move to the location.

Also, according to the invention, a method is described that provides aninteractive simulation of a virtual environment to a user at a simulatorstation. The method comprises storing physics data on a computeraccessible memory coupled with a physics engine. The physics datadefines physics parameters of a plurality of virtual objects in avirtual scene, and includes position data reflecting respectivepositions of the virtual objects in the virtual scene. The physicsengine is passed data identifying at least one of the virtual objectsand defining a value of force or torque, the physics engine modifies thephysics data in the memory so that the position data of the virtualobject in the memory reflects a modified position that corresponds to aresult in the virtual environment of the force or torque acting on thevirtual object pursuant to physics rules of said physics engine. Themodified position data is obtained and stored said so as to beaccessible to a computerized viewer. An image is rendered correspondingto a view of the virtual scene using the computerized viewer, and theview includes a view of at least a portion of the virtual object in themodified position.

According to another aspect of the invention, a system is provided forproviding distributed simulations to one or more users. The systemcomprises a first computer system linked for communication over anetwork and having a data storage device storing data and a display. Thedata storage device stores data defining a virtual scene having a numberof virtual objects therein each having a respective visual appearance,position and orientation defined by the data, the computer systemincludes a viewer generating real time imagery from the data anddisplaying the imagery on the display, the imagery corresponding to arendered view at the time the image was rendered of the virtual scene asdefined by the data. A second computer system has a data storage device,the data storage device storing data defining the same virtual scenewith the same virtual objects as defined by the data on the firstcomputer system. The second computer system executes a programdetermining movement of a controlled virtual object in the virtualscene. The second computer system further has a master synchronizertransmitting over the network route data for the controlled virtualobject indicative of a point in the virtual scene and a future point intime by which the controlled virtual object is directed to reach thepoint. The first computer system has a slave synchronizer receiving theroute data, and responsive thereto, causes the data stored therewith tobe modified so that in the data stored at the first computer system, thecontrolled virtual object moves to the point in the virtual scene by thepoint in time.

Other objects and advantages of the invention will become apparent fromthis specification, and the scope of the invention will be set out inthe claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of an embodiment of a distributedsimulation system according to the invention;

FIG. 2 is a schematic diagram of physics farm logical architectureaccording to the invention;

FIG. 3 is a more detailed schematic diagram of the physics based systemof the invention;

FIG. 4 is a diagram of a DIS network processing theory;

FIG. 5 is a graph illustrating the possible erroneous results of deadreckoning.

FIG. 6 are comparative diagrams showing how the prior art extrapolationmethods allowed for extrapolation through solid objects, while thephysics approach of the invention does not;

FIG. 7 is a diagram illustrating the physics based method for correctingfor latency during collisions;

FIG. 8 is a flowchart showing the general steps of physics processing;

FIG. 9 is a diagram illustrating the processing of physics data toproduce desired displays at two distributed simulators;

FIG. 10 is an event trace of an exemplary collision in a systemaccording to the invention;

FIG. 11 shows diagrammatically examples of data message formats forphysics data transmitted between the physics engines;

FIG. 12 is an event trace showing creation of an entity and anapplication of force to an entity;

FIG. 13 is a diagram comparing operation of the physics based system ofthe invention with the current military system;

FIG. 14 is a more detailed diagram of the components of the physicsfarm;

FIG. 15 is a diagram of the operation of the physics manager;

FIG. 16 is a diagram of the resource management operation of the physicsapplication manager in a physics based system according to theinvention.

FIG. 17 is a diagram of the physics application manager communicationsin resource management in a system according to the invention;

FIG. 18 is an illustration of the problems of coordinate conversion andthe elimination of the problem in the system according to the invention;

FIG. 19 is a diagram of physics networking and entity management in thesystem;

FIG. 20 is a diagram of the physics based system operation forintegration of a vehicle simulator with Semi-Automated Forces (SAF) orComputer Generated Forces (CGF);

FIG. 21 is a diagram of data flow between a physics engine and an SAFapplication.

FIG. 22 is a simplified block diagram of a simple example of a physicsbased system constructed to test feasibility.

FIG. 23 is a diagram of a synchronizer system according to theinvention.

FIG. 24 is a diagram illustrating the synchronizer system of theinvention as implemented for a SAF vehicle.

FIGS. 25 and 26 are illustrations of exemplary object movements that canproduce divergent solutions in the scene data of simulators that areseparated by a network connection that introduces latency.

FIG. 27 is an illustration of remedial action taken by the slavesynchronizer of the invention to circumvent an obstruction.

FIG. 28 is an illustration of another remedial action that may be takenby the slave synchronizer of the invention to circumvent an obstruction.

FIG. 29 is a diagram of a desired route for a client object vehicle inthe virtual world of the simulation scene data.

FIG. 30 is a diagram of the synchronizer control of the simulatedvehicle where delay occurs in the prescribed route.

FIG. 31 is a schematic diagram of the speed control loop for a vehiclein simulation.

FIG. 32 is a schematic diagram of the steering control loop for avehicle in simulation.

FIG. 33 is a schematic diagram of the heading control loop for a lifeform in simulation.

FIG. 34 is a flowchart of the general operation of a slave synchronizeraccording to the invention.

FIG. 35 is a schematic diagram of the physics-based environmentgenerator infrastructure system according to the invention.

FIG. 36 is a schematic diagram of the synchronizer concept of theinvention in a physics-based environment generator (PBEG) infrastructuresystem.

FIG. 37 is a schematic diagram of another aspect of the synchronizerconcept of the invention in a PBEG infrastructure system.

FIG. 38 is a schematic diagram of a plug-in aspect of the synchronizerconcept of the invention.

FIG. 39 is a schematic diagram of a PBEG infrastructure system accordingto the invention.

FIG. 40 is a detail view of a portion of the schematic diagram of FIG.39.

FIG. 41 is a schematic diagram of the flow of data in an initializationphase of the operation of a PBEG infrastructure system according to theinvention.

FIG. 42 is a schematic diagram of the flow of data in the creation of aphysics-based virtual object in a PBEG infrastructure system accordingto the invention.

FIG. 43 is a diagram of the flow of data in a client system using a PBEGinfrastructure system according to the invention.

FIGS. 44 and 45 are schematic diagrams of the flow of data to and from aPBEG exercise management application according to the invention.

FIG. 46 is a timeline diagram illustrating the transfer of control of aclient object from a master synchronizer at one computer system to amaster controller at a second computer system.

DETAILED DESCRIPTION

As best shown in FIG. 1, a simulation system according to a preferredembodiment of the invention has a number of simulation stations 3 eachlinked to a network 5, either a LAN or the Internet. Instructor station7 is also linked to the network 5, as is an auxiliary client computersystem 9, the operation of which will be discussed below.

Each simulation station is a computerized system that supports aninteractive involvement of a respective user. The simulator stations 3each include a display device 11 of some sort, ranging from a projectionsphere to a head mounted display. The simulation stations also includean interactive device that allows a user to input data to the simulator,and where a vehicle is being simulated, this usually takes the form of asimulated cockpit or driver's seat 13 that inputs data but is alsocontrolled by the simulator computer system 15 to afford realism to thesimulation, as with functioning gauges and indicators on the controlpanel of the vehicle. The simulator includes a computer system 15 thatmanages the operation of the interaction with the user, as is well knownin the art.

In addition, the simulator station 3 includes a computerized imagegenerator 19 that accesses scene data stored on a data storage system21. The scene data stored includes data defining effectively everyvisible virtual object in a virtual “world” or environment in which thesimulation takes place. This scene data includes all information neededto determine the appearance of the objects in the virtual world,including the location, shape, size, orientation, color, texture, andany other visible characteristics that the object may have. Imagegenerator 19 generates video imagery from the scene data by renderingviews of the scene data for the point of view of the user from theuser's virtual location in the virtual world.

The scene data, of course, can change as objects move in the virtualworld, although at start-up of the system, all simulator stations 3 havethe same scene data in their mass data storage 21, because they all arein the same world.

Simulator stations 3 also have a physics engine component 23 thatcommunicates with the network or Internet 5. This physics enginecomponent 23 includes at its heart one or more physics engines, whichare preferably physics processor units (PPUs), which are dedicatedhardware devices with supporting circuitry that can process largeamounts of data defining physical parameters of virtual objects in asimulated three dimensional environment world, and determineinteractions for those objects expressed as forces, accelerations,position, velocity and various constraints on an objects movement due tocontacts with other objects in the simulated world.

Particularly preferred for use in the physics engines are those based onthe physics processors sold under the names PhysX or NovodeX by acompany named Ageia, having a place of business at 82 Pioneer Way, Suite118, Mountain View, Calif. 94041. Other dedicated-hardware physicsprocessor systems may also be used advantageously with the invention. Atpresent the computational power of any software system is so far belowthat of the dedicated hardware physics engines, which can process on theorder of thousands of times more objects, that a software system, interms of present technology at least, would have at best a very limitedutility.

The physics engine component 23 also includes a memory system,preferably a RAM memory, that stores physics environment data definingthe physical attributes of the virtual objects in the virtualenvironment, including a detailed model of the ownship vehicle in avehicle simulator, expressed as an assembly of constituent virtualparts. Each object or entity has physical attribute data and locationdata stored in this memory, including data indicating the geometry ofthe entity, its mass, its center of gravity, its inertia, itsorientation, its density, and any joints or contacts that may connect itwith any other objects. The physical attributes are used by the physicsengine to determine movement of the virtual objects based on physicalrules of the physics engine, which emulate the physical laws of the realworld.

The physics engine component 23 preferably includes a physics manager aswell, which controls which physics engine does the physics calculationsfor any specific object. For example, the objects making up the ownshipof the simulator are normally assigned to the physics engine of thatsimulator, because changes in the physical data are especially importantto that particular simulator, and latency in reporting movement in thoseparts is very undesirable.

Objects in the virtual world that are near the virtual position of theuser in the simulation system, such as a building next to which ahelicopter simulated by the simulation station is landing, are alsopreferably processed at the simulator station physics engine, becausetheir closeness makes latency and the resulting inaccuracies in displaymore undesirable. On the other hand, objects making up a distantbuilding that has little if any effect on the user's view areappropriately processed in a different physics engine, because latencydoes not have a significant negative effect on the user's view. Oneexception to this is that if a weapon subsystem is being simulated, adistant object is processed by the local physics engine if the object isbeing targeted, as the object though distant is critical to the user'sview.

The collection of all of the physics engines connected with the network5 and their managers is referred to as a “physics farm” as it is adistributed system the parts of which cooperate to share the physicsdata and computational workload for the simulation.

In operation, each physics engine only acts in response to a forceapplied to an object for which it is responsible. When that happens, ifthe object is subjected to enough force to cause it to move, the physicsengine will calculate its movement and change the local scene dataaccordingly. It will also broadcast over the network 5 to all othersimulator stations 3 or physics engines in the system a messageindicating the amount, location and time of the force applied to theobject. Those other stations 3 will then receive the message and updatetheir own copy of the world's physics data by applying the physicalrules to this data message, with other local processing as will bediscussed below to compensate for latency in the transmission.

If the force moves the object so that it contacts another object, thenthe physics engine resolves the collision to determine where the twoobjects wind up when it is over. In addition, a slightly differentcollision message of data is sent out identifying the two objects thatcollided. The other physics engine that is responsible for the otherobject contacted also will issue a collision report. The scene data isupdated by each physics engine to reflect the end position of the twoobjects pursuant to the physics engine rules.

Referring again to FIG. 1, in addition to the simulation stations 3,additional physics engines 25 can be provided in a computerized system 9also connected with the network. These physics engines can assist incomputation for the simulation stations 3, but can also provide forphysics processing of other objects moved in the virtual world bysomething other than the users, such as by applications 27 that accessthe physics engines 25 through physics-based simulation API 29. Examplesof such applications are computerized processes that provide anenvironmental influence, such as a hurricane-force wind, which producesforce on a number of objects; and intelligent behavior or SAF modeling,which provides for movements of computer-controlled forces that operateessentially on their own independently of the users or in reaction toevents in the simulation, for example, as enemy forces, support forces,or something else in the simulation.

Administrator or instructor IOS stations is a computer system that canaccess any physics data in the system to set up a desired trainingscenario, as will be discussed below in greater detail.

FIG. 2 illustrates a general system approach for the logicalarchitecture of a physics-based simulation in accordance with onepreferred embodiment. The infrastructure is optimized forsynchronization and time delays by providing an optimized network andservice layers. This infrastructure includes software managementservices, physics based APIs and network protocol that is unique tophysics based distributed simulation. In this embodiment, the logicalarchitecture is broken down into four primary layers 110, 120, 130, and140.

The physics farm communications layer 110 is used to communicate andexchange data between the participating physics engines 121, all ofwhich are linked via the communications backbone network to receive andtransmit data to and from the other physics engines 121 in the system.

The physics farm layer 120 enables a physics-based simulation to beperformed in a distributed environment. The physics farm manager 123dynamically assigns objects in the simulated environment to one of thephysics engines 121 for physics processing, as discussed elsewhereherein. The physics based simulator API interfaces the physics farm datawith the other software applications components of the simulationsystem.

The physics-based simulation application layer 130 is comprised of theone or more applications, or simulation components/entities,participating in the distributed physics based simulation; e.g., typicalcomponents/entities might include one or more helicopter, aircraft orground vehicle simulation stations, computer models that control groundor life form behaviors, managing environment objects, occulting orline-of-sight results, SAF behaviors, or similar modeled functionalitiesexecuted on connected computer systems that influence the virtual worldof the simulation.

The physics-based simulation training layer 140 enables the introductionof training artifacts typically employed in a training environment intoa physics-based simulation. The physics-based simulation training layeris desirable but not strictly necessary to an operating distributedsimulation system, and is an alternate embodiment for systems whereclosely supervised or very flexible training is desired. In anotherembodiment the physics-based simulation training layer is replaced witha layer that performs the specific system-wide administrative functionsrequired by the particular purpose of the system, such as, in amultiplayer video game environment, layer 140 can process overallmanagement of player accounts, statistics, etc.

The physics farm communications layer 110 provides the communicationsbackbone for the physics-based simulations contemplated by the presentinvention. Physics-based simulations introduce actions and reactionsthat can be highly sensitive to time and latency issues that have notbeen adequately addressed by previous approaches using DistributedInteractive Simulation (DIS) protocol or High Level Architecture (HLA)based systems.

In one aspect of the invention, timing and latency issues are reduced oreliminated by determining the resulting positions, velocities andaccelerations when two objects collide using the physics engines,instead of using the applications themselves to represent the collidingobjects, as in traditional simulations.

One approach used by current distributed simulation training systems isto incorporate threshold techniques to minimize network traffic; i.e.,applications responsible for their own objects won't send datareflecting current positions, velocities and accelerations until somepredetermined position, orientation or time delta has exceeded aspecified threshold.

Such threshold techniques disadvantageously require the local orreceiving application/simulation to extrapolate position and orientationover time, i.e., in the period of time that new data has not yet beenreceived from the remote application/simulation. The result ofimplementing such threshold techniques is that the same object may be ina different position and/or orientation in one simulator than it is inanother participating simulation. The physics based outcome of acollision may produce different results in different distributedparticipating simulations. Because time and latency may also presentissues in the physics-based simulations creating potentialinconsistencies, new methods employed by certain preferred embodimentsof the present invention are used to resolve conflicts and maintainconsistency.

In one aspect of the invention, latency is minimized and communicationsbandwidth is maximized using the physics farm communications layer 110.Preferably, the physics farm communications backbone 111 is a highbandwidth network that is highly optimized to the protocols used byparticipating physics engines. In one preferred embodiment, data ispackaged, prioritized and shared between physics engines in order toprovide realistic and consistent results. According to another aspect ofthe invention, the physics farm layer 120 is used to obtain realisticand consistent results.

The physics farm layer 120 is a collection of capabilities/sub-layersincluding, preferably: the physics farm application programminginterface (API) layer 125; the physics farm management layer 123; andthe physics engines 121 (referred to interchangeably throughout as “thephysics farm”) participating in a physics-based simulation. Theindividual physics engines 121 managed by the physics farm managementapplication 123 are preferably treated as a pooled resource where somephysics engines may be loosely coupled and others may be tightly coupled(i.e., embedded within a simulation application).

The capabilities of the physics farm layer 120 are preferablydistributed with data exchanged via the physics farm communicationslayer, or backbone. The physics engines are preferably instances of thephysics algorithms used to model the physics environment, deployed ineither dedicated hardware (e.g., physics processor unit, or PPU) or oneor more software routines. Physics-based objects (i.e., objects forwhich physics based computations are required to allow the object tointeract within the distributed simulation) are registered, orinstantiated, with the responsible physics engine. Physics-based resultsare then preferably produced based on the objects' attributes, such asposition, velocity, mass, dimensions, and the like.

Physics farm management is responsible for coordinating and managing theactivity between participating physics engines. According to one aspect,the physics farm management layer preferablyarbitrates/manages/coordinates which physics engine will be responsiblefor modeling a specific object or request. The physics farm managementlayer 123 preferably balances network/computational load(s) and resolvesconflicts in order to maintain consistency throughout the distributedsimulation.

The physics farm management capabilities preferably reside with eachinstance of a physics engine, whether loosely or tightly coupled, andwill use the physics farm communication backbone to exchange or sharedata. Preferably, each simulation application employs logic allowing thesimulation applications to access the physics farm capabilities usingthe physics farm API to make requests for services to be performed or toimplement callback routines to receive results or updates to thedistributed simulation state.

Advantageously, the physics farm API provides applications with aninterface to the physics engines as well as the physics based simulationenvironment. In addition, the physics farm API hides the details andnuances involved with distributing and balancing the load across theparticipating and distributed physics engines.

In accordance with yet another aspect of the invention, thephysics-based simulation application layer 130 implements the simulationapplication logic. Preferably, the logical architecture is designed suchthat any application logic can reside in the physics-based simulationapplication layer (FIG. 3 illustrates some representative simulationapplications). Preferably, those simulation applications requiring ordesiring physics-based simulation use the physics farm API to create,delete, modify and obtain results of actions to be performed byspecified objects. Actions can include instruction to move to specificlocation, instructions to apply a force or acceleration, and the like.Similarly, actions can include instructions to trigger a model toperform an abstract operation, such as to detonate or explode, injectingan impulse force into the physics based simulation. Preferably, anycombination or sequence of actions can be utilized.

Where instructor involvement in the training is provided, the physicsbased simulation training layer 140 can be employed to allow theintroduction of training artifacts into the physics-based simulationtraining environment. This layer 140 may be referred to interchangeablyherein as the Next Generation Training (NGT) infrastructure. Preferably,NGT supports a distributed training environment based on commercial webbased technologies such as, e.g., XML, Java, Web Services, and the like.

The physics-based simulation training layer 140 may include componentsfor instructor operator services (IOS), brief/debrief, exercise andresource management capabilities, or the like. The functions of the IOScan include: exercise planning, objectives, performance measures,initialization, monitoring, controlling the progression, After ActionReview (AAR) of the exercise, etc. For example, the brief/de-briefcapabilities may be utilized by the IOS to brief the training audienceprior to the start of an exercise and de-brief the training audience onthe outcome of the exercise.

In a particularly preferred embodiment, the IOS is an instructor stationwith a data entry interface and a display. From the IOS station, aninstructor can access physics based attributes of objects to initializeor modify the attributes of one or more aspects/objects of the physicsmodel during the exercise. Advantageously, using a physics-basedapproach in accordance with one aspect of the invention provides a moreflexible and simpler approach to insertion of malfunctions and controlof the distributed simulation. For example, an instructor can accessthrough the IOS station the physical attributes a support for an engineof a vehicle to cause it to break, creating a custom-designed trainingscenario.

FIG. 3 further illustrates a physics-based simulation block diagram 201in accordance with the present invention, and the interconnection ofcertain components of a physics-based simulation training system inaccordance with one preferred embodiment.

Simulation applications 210 can generally have two connections, oneconnection to the physics farm 220 through the physics farmcommunications backbone 230 and another connection to the NGTinfrastructure 240. The connection to the NGT infrastructure can beachieved by using an application bridge 250, where the purpose of thebridge would be to preferably translate application specificrepresentations/data structures to a common NGT representation/datastructure.

Additionally, gateways 260 are employed to convert between disparateprotocols and data structures; e.g., DIS to HLA, NGT to DIS, toaccommodate distributed simulations that may include legacysystems/networks. Also the incorporation of commercial game engines 270is shown, these engines having embedded (i.e., tightly coupled) physicsengines, into the distributed physics-based simulation.

In FIG. 3, the game engine server component is connected to the physicsfarm communications backbone 230 and the individual game clients areconnected to the game engine server (most present day multiplayer gamingsystems employ a client-server network architecture). As discussed inmore detail below, the physics farm communications backbone ispreferably optimized for physics-based requests and callbacksfacilitating communication of physics model/state data between physicsengines. Additionally, one or more physics engines in the physics farmcan be assigned responsibility of performing physics-basedcomputing/processing for applications 210 lacking their own physicsengine.

The computer architecture of the simulation stations 3 is schematicallyillustrated in FIG. 19. FIG. 19 shows the physics networking and entitymanagement approach employed in a preferred embodiment of the presentinvention. The physics engine utilized in computing the remote entity'sstate can either be a locally embedded, tightly integrated physicsengine or a loosely coupled physics engine that is allocated to thephysics farm and managed by the physics farm manager. In FIG. 19, twosimulation applications 410, 420, e.g., simulation stations 3,participate in a distributed interactive simulation by communicatingacross the physics communication backbone 400. The physicscommunications backbone 400 is used to communicate or sharephysics-based state data (e.g., forces, accelerations, orientations,collisions, position, etc.) between physics-based simulationapplications 410, 420. This backbone 400 may be a communicationsprotocol system using the network 5 that allows communications betweenthe physics engines.

In the preferred embodiment, host processors' system clocks aresynchronized using, e.g., the standard network time protocol (NTP), andtime-stamped physics-based state data is preferably communicated overthe physics communications backbone 400 for each physics-based entity.

Each simulation application preferably includes a local physics farm APIinterface 413 and a physics manager 414. In certain simulationapplications, the physics engine may be embedded (as depicted in thesimulation applications 410, 420). However, as shown in FIG. 20 anddiscussed in greater detail below, the simulation application mayinclude only a physics farm API and physics manager. In such anapplication, without an embedded physics engine, the application can usethe physics farm to interact with the physics-based environment.Specifically, the physics manager can request, and be allocated, theservices of one or more of the physics engines in the physics farm. Inlight of the fact that the physics engine may or may not be embedded,the physics farm API is preferably configured to encapsulate both thephysics manager and the physics engine since the physics manager shouldbe capable of intercepting local changes to forces when they areapplied.

As depicted in the illustrative example of FIG. 19, the local physicsmanager, e.g., 414, notifies the remote physics manager, e.g., 424, ofany new, modified, removed or deleted entities, and vice versa. If acollision event is generated by the physics engine, the physics manageruses the collision event as an indicator that corrective action may berequired. If a collision event occurs for the local entity, the localphysics manager preferably sends only those new forces resulting fromthe collision event. Upon receipt of a collision indicator, thereceiving physics manager(s) preferably apply any needed corrections forthe notifying remote entity. Through use of the physics-based approachillustrated in FIG. 19, extrapolation by dead reckoning is no longernecessary since the physics engine(s) are continuously applying forcesacting upon remote entities.

The problems of “dead reckoning” extrapolation, which has been used inpast distributed systems to try to compensate for latency in the networkare explained in FIGS. 4 and 5. In FIG. 4, the current approach ofmilitary simulations employing DIS protocols is illustrated. Theparticipating simulation 150 depicts a high fidelity, dedicatedsimulator 151, e.g., helicopter, and includes a rendering engine 153,and an engine to incorporate remote entities 155. Another participatingsimulation 157 depicts an SAF application embodying a behaviors engine159, a vehicle dynamics engine 122 and an engine to incorporate remoteentities 163. The SAF application is a program that that models avehicle or other entity, such as a military unit, that moves on its own.Each participating simulation 150,157, etc. publishes or communicatesits entity (i.e., virtual object) state (e.g., position, orientation,linear velocity, linear acceleration, angular velocity, etc.) across theDIS network 165.

Each participating simulation 150, 157, etc., locally computes theposition and orientation of the entity states of remote participatingsimulations until the next entity state update from the remote entity isreceived. The remote entity's state is computed locally using the remoteentity engine 155, 159 by performing linear extrapolation, known in theart as “dead reckoning”.

The basic idea underlying the use of dead reckoning algorithms is thatrather than each and every entity sending frequent state updates, eachentity's remote entity engine 155, 159 estimates the location of theremote entities based on their last reported entity states. A deadreckoning algorithm is typically specified for each entity and eachentity publishes its entire entity state only when its actual statediffers from that predicted by the dead reckoning algorithm by somedelta, i.e., a threshold value is exceeded. As a result, participatingsimulations render imagery displayed to their respective users based onthe dead reckoned remote data for remote entities while using actualdata for controlling “ownship”/SAF vehicle dynamics 151, 161. Moreover,the SAF vehicle dynamics used in present day military simulations employsimplified kinematics models.

As can be seen in FIG. 5, the use of dead reckoning algorithms, i.e.,linear extrapolation, can result in inconsistencies between local andremote entities in representing the state of the simulation. Thisresults from the fact that the actual position of a remote entity istypically not linear. As best shown in FIG. 6, another problem isencountered in the dead reckoning method. Simply put, the lineardead-reckoning extrapolation can extrapolate the object to a locationthat it could not reach in reality, such as the far side of a wall orother barrier.

The physics-based approach according to the present invention does notuse dead reckoning algorithms but instead specifies that each simulationapplication locally compute the entity state of physics-based remoteentities using a physics engine. The physics information for the movingobject is received by the physics engine of the relevant simulatorstation 3, and the movement of the object under the effect of thedefined forces is projected, taking into account the possibleobstruction caused by other objects in the scene. This prevents anyextrapolation “through” another object. The physics engine willdetermine the interaction of the objects and process the collision as anevent, but the object will not be extrapolated to an impossiblelocation.

Physics-based modeling readily provides for the detection of, and thecomputation of, resulting forces due to, collision events. Since, asdiscussed above, dead reckoning is not used by the physics-basedapproach of the present invention, but instead simulation applicationslocally compute the state of each remote physics-based entity, collisionevents can be readily detected. Accordingly, any resulting changes inthe remote entity's physical state may also be readily computed.

Nonetheless, collision events handled by the preferred physics-basedapproach may also suffer from latency issues associated with networkcommunications of physics-based data.

FIG. 7 illustrates at least one of the issues that may be associatedwith latency in a physics based system, and also illustrates a preferredapproach that may be used to compensate for such latency. In FIG. 7, alocal host 170 detects a collision event of entity 171, and 171 a and171 b represent the positions of entity 171 before and after thecollision event, respectively. According to this embodiment, the localhost 170 broadcasts notification of the collision event the moment itoccurs. Simultaneously, a remote host 180 also detects or computes thecollision event, since it is using a physics engine to compute thebehavior of the remote entity 171. However, due to one or more technicalconsiderations, e.g., a latency-created difference in the relativepositions of object 171 and the object 173 with which it collides,potentially differences may appear between a local host's 170 computedstate and position 171 b of a local entity and one or more remote host's180 computed state and position 171 d for that same entity, which istreated as a remote entity on a remote host.

FIG. 7 illustrates the difference that can occur when the locally andremotely computed positions or states differ and a method forreconciling or de-conflicting the results in the event of suchdifference. To keep the distributed simulation environment synchronizedto a certain predetermined, acceptable level, the local host 170broadcasts the collision event of a locally hosted entity, including theresults of such a collision event on the local entity's state.Preferably the entity state information broadcast as a result of thecollision event includes a time-stamp of a system time for thecollision, or some other useful temporal indicator, such that, when itis received by the remote host 180, the remote host can compute anynecessary adjustments that might be required in order to reconcile theentity's state as computed by the remote host and the entity's actualstate as computed by the local host.

As can be seen in FIG. 7, at the point in time when the entity's actualstate is received by the remote host, the remote host would have alreadyaltered the state, i.e., location, of the remote entity based on its ownphysics-based computations, and potentially a few frames would have beengenerated during this latency lag period, resulting in the object'sposition being calculated at point 171 c. The remote host 180 thenprojects out in time some increment to compute the necessary correctionsthat must be applied in order to converge the remotely computed state ofthe entity with the entity's actual state. This corrective path,illustrated in the graph, is smoothed and timed so as to preventcorrective jerking of objects in the simulation.

Collision events, in accordance with one aspect of the invention, arepreferably dealt with according to the following approach. Within thelocal simulation application, based on the contact point of a force on agiven entity (e.g., box, features, etc.), the constraint forces arecomputed and sent to the remote simulator(s). Using the constraintforces transmitted by the local simulation application, accelerations,velocities and position can be computed within the remote simulators'physics engine(s). Preferably, forces are only sent when they exceed acertain predetermined threshold value. For example, if we assume that ina simulation, for example, there is an IED explosion that is aninstantaneous impulse, then the force is broadcast to all othersimulators only one time. Advantageously, this approach does not requirethe transmission of continuous updates reflecting position, velocitiesand accelerations. Further, complex extrapolation or dead reckoningcomputations are not needed.

FIGS. 8 and 9 illustrate the physics engine computations performedaccording to a preferred embodiment. Essentially, the process involvessolving for constraints and then generating the information needed toupdate an entity's state (e.g., accelerations, velocities andpositions). The forces and accelerations are determined with six degreesof freedom, meaning, linear forces applied at the three orthogonalspatial axes and rotational component forces about each of those axes.

As shown, first at step 210, contact points are determined using surfacenormals. Next, in step 220, the constraint forces are computed based onthe determined contact points. Then, at 230, accelerations, velocitiesand position are integrated to obtain a result in terms of currentposition of the object. The physics engine continuously carries out thiscomputation at each time step for each physics-based entity registeredwith the engine, and the scene data of the object in the associatedsimulation station 3 is updated based on the calculated position of theobject.

FIG. 10 illustrates an event trace based on a collision event using thephysics-based approach according to a preferred embodiment of theinvention. Depicted by way of example are two simulation applications810,820, each including a respective physics farm API 811, 821, arespective physics engine 812,822 and a respective physics manager 813,823, with the two simulation stations connected by the physics farmcommunications backbone 830.

The event trace for the exemplary collision event proceeds as follows.

1. For an entity (i.e., virtual object) hosted by simulation application810 (i.e., an instance of a participating physics engine owning theentity simulation), a collision is detected by the physics engine 812and reported to the local physics manager 813. Typical informationreported includes, e.g., entity ID, colliding entity ID (entity withwhich collision occurred), collision position and collision time.

2. The physics engine 812 also reports the same information to the localphysics farm API 811, which in turn passes the information along to thelocal simulation application 810 for updating the local scene data toreflect the end location of the virtual object or entity.

3. The local physics manager 813 broadcasts over the physics farmcommunications backbone 830 information related to the collision event,including, e.g., entity ID, colliding entity ID, collision time and thenew forces computed by the local physics engine 812 as a result of thecollision.

4. The remote physics manager(s) 823 receives this information and usesthe information to synchronize and de-conflict the results broadcast bythe simulation application responsible for the entity which was thesubject of the collision with its own results that it has been computingsimultaneously on its own physics engine.

5. The physics manager transmits the entity's position and other statedata to the physics farm API 821 of its simulator station, to be passedon for incorporation into the scene data stored therein

FIG. 11 illustrates a preferred embodiment of message formats forphysics data transfers among the physics engines over the physics farmcommunication backbone. There are three different data formats orpackets used for transmitting the physics data.

When an object or entity ids created, the event is broadcast over thephysics farm backbone with a data packet 271 containing the entity ID,the entity type (defining geometries and other physical attributes ofthe entity), the time of creation in system time, the initial positionof the entity, and the initial forces on the entity, defined in the sixdegrees of freedom discussed above.

When a force is applied to an object sufficient to cause it to moveabove a threshold limit, an applied force data record or packet 273 istransmitted, containing the entity ID receiving the force, the time ofthe force, and data defining the force in six degrees of freedom.

When a collision occurs, a collision data or data packet 275 istransmitted to the other physics engines over the physics farm backbone.The collision data includes the entity ID of the entity hitting anotherobject, the entity ID of the other object that is hit, the time insystem time, the location of the collision, and the forces resultingfrom the collision.

FIG. 9 illustrates an event trace based on the creation and update of aphysics-based entity. Depicted by way of example are two simulationapplications 910, 920, each including a respective physics farm API 911,921, a respective physics engine 912, 922 and a respective physicsmanager 913, 923, with the two stations connected by the physics farmcommunications backbone 930.

The event trace for the exemplary collision event proceeds as follows.

1. For an entity hosted by the local simulation application 910 (i.e.,an instance of a participating physics engine owning the entitysimulation), the simulation application 910 communicates to the localphysics farm API 911 a request to create an entity, providinginformation such as, e.g., kind of entity, entity ID, initial time,initial position and initial forces.

2. The physics farm API in turn then communicates a request to the localphysics engine 912 to load the corresponding physics model with thegiven entity attributes, position and forces and notify the localphysics manager 913 of the creation of the entity and its correspondingcreation data.

3. The local physics manager 913 broadcasts the creation of the entityand its associated creation data across the physics farm communicationsbackbone 930.

4. The remote physics manager(s) 923 receives the notification of thecreation of the new entity on simulation application 910.

5. The remote physics manager(s) 923 proceeds to instantiate the entitywithin its local physics engine; i.e., instruct its own physics engine922 to load the physics model/data for the new entity.

6. The remote physics engine 922 then notifies the remote physics farmAPI 921 of the creation of the new entity.

7. The remote physics farm API 921 in turn notifies the remotesimulation application 920 of the creation of the entity.

Similarly, as shown in FIG. 9, an exemplary application of forces to anentity proceeds as follows:

1. The local simulation application 910 communicates to the localphysics farm API 911 the request to apply a set of forces to an existingentity owned by that simulation application; typical informationincluded in the request might include, e.g., entity ID and set of forcesto be applied.

2. The local physics farm API 911 then instructs the local physicsengine 912 to apply the forces and at the same time notifies the localphysics manager 913 of the request to apply the new set of forces.

3. The local physics manager 913 in turn broadcasts the application ofthe new set of forces to the entity across the physics farmcommunications backbone 930.

4. The remote physics manager(s) 923 receives the broadcast andcommunicates the information to the remote physics engine 922 so thatsimulation of that remote entity can now be carried out using the newset of forces.

In the physics based system of the present invention, frequent entitystate updates and consequent frequent extrapolation by dead reckoningare not required. FIG. 13 illustrates this with a side-by-sidecomparison of the current approach used in military DIS basedsimulations with the approach according to a preferred embodiment of thepresent invention. The left side of the diagram of FIG. 13 depicts thelocal host activities and the right side depicts the remote hostactivities. In the upper half of the diagram depicts the currentmilitary DIS approach and in the lower portion depicts the approachaccording to a preferred embodiment of the present invention.

As can be seen from FIG. 13, in the current military DIS approach, thelocal host updates the actual position, velocities and accelerations fora locally owned entity fairly frequently, and then performs deadreckoning of the local entity to check for any excursions beyond thepredefined threshold value. Only when the local dead reckoningcomputations exceed the predefined threshold value(s) does the localhost broadcast an update across the DIS network. Updates are alsobroadcast upon the occurrence of a collision event. Therefore, in theabsence of entity updates, the remote host continues to compute theentity's state based on the dead reckoning algorithms and needs toexecute correction routines when an entity update is received, i.e.,when the local/owning host determines that the dead reckoningcomputations no longer accurately represent the entity's state based onsome predefined threshold/tolerance value.

In contrast, in the physics-based approach of the present invention, thelocal host does not perform any dead reckoning threshold computations,and only updates the entity's state when forces have changed or when acollision event is detected. Similarly, the remote host does not performany dead reckoning/extrapolation of the local entity's state since ithas the same physics model of the entity loaded and is simulating itsbehavior based on the same set of applied forces and collision events.

Modeling of the ownship for vehicle simulator stations has in the pastusually used scripts triggered by events or the instructor, and theseprovide limited scenarios for training. According to the preferredembodiment of the invention, however, the simulation station has adetailed physics-based model made up of the entities that together formthe user's ownship. As such, the physics engine at the simulationstation 3 is preferably used to perform physics processing on theconstituent parts of the vehicle being simulated. The result is that thephysics engine provides a realistic and very flexible simulation of thedynamic operation of the vehicle. Component parts may be damaged byother entities in the virtual world, or influenced by an IOS commandentered for training purposes, in which case, the physics engine causesthe vehicle in simulation to perform (or fail) similarly to the realvehicle with a similar type of damage.

The simulation based on physics processing of the virtual entity modelof the ownship may also be used to supply simulated data to thesimulator cockpit or control panel, such as, in an aircraft, providingdata defining an artificial horizon or altimeter gauge.

In addition, the physics engine of the simulation station generatesforce or collision data reports in the processing of the ownshipconstituent entities or virtual objects. That physics data is publishedas events on the physics farm backbone to the other physics engines,which use the force or collision data to update their associated copy ofthe world physics data, which includes the constituent entities thatmake up the ownship of each simulator station in the system. As aresult, not only is the physics data for the ownship components used forsimulating operation at the associated simulator station, but it is alsoused to render the views that other users on the system have of thevehicle. Because these views correspond exactly (except for any latencyissues) with the exact operational simulation of the vehicle, reality ofthe scene is enhanced, as well.

FIG. 14 illustrates an embodiment according to the present invention ofthe physics farm. The physics farm can be conceptualized as the dynamicpooling of all physics-based resources participating in a distributedsimulation. The pooling of physics-based resources is made possible byemploying certain physics farm components needed to effectively managethe distributed simulation; e.g., physics farm API, custom physics APIsfor interfacing legacy applications, physics managers and physicsengines.

In FIG. 14 a dedicated, high fidelity simulator representation 1010(e.g., a helicopter flight simulator) is participating in a distributedphysics-based simulation with a more traditional SAF application. Thesimulator 1010 includes its own rendering engine 1011, a physics farmAPI 1012, a physics manager 1013 and an embedded physics engine 1014. Asimplified representation of an SAF application 1020 is also depicted.

The SAF application 1020 includes an SAF physics API 1021, a physicsfarm API 1022 and a physics manager 1023. A simplified representation ofan intelligent behaviors application 1040 is also shown, having anartificial intelligence (AI) physics API 1041, physics farm API 1042 anda physics manager 1043.

In generic form, any simulation application can be adapted to work withthe physics-based network, not only simply participating, but also usingthe physics-based processing to govern the behavior of entities hostedby its own simulation. For example, a physics farm application component1030 can include an SAF physics manager application, an intelligentbehaviors physics manager application, an environment managerapplication or vehicle physics manager application.

In order to effectively participate, communicate and utilize physicsfarm resources, the physics farm application component applicationincludes a physics farm API 1031, a physics manager 1032 and a physicsengine 1033. Alternatively, the physics farm application component doesnot include its own embedded physics engine (not shown) but instead usesa physics engine in the physics farm 1045, either a tightly or looselycoupled physics engine available as a pooled resource. Another componentincludes a physics manager application 1050 for managing the resourcesof the physics farm. The physics manager application includes a physicsfarm API to communicate and manage the physics farm components.

FIG. 15 illustrates the interoperation of the physics managerapplication 1100 with a simulation application 1110, or generically aphysics farm application component, according to a preferred embodiment.In operation, a participating simulation application 1110, orgenerically a physics farm application component, notifies the physicsmanager application 1100 via the physics farm communication backbone1140 of its participation in the simulation. The physics managerapplication 1100 then registers/un-registers (if a physics managernotifies the physics manager application that it is terminating itsparticipation in the simulation) the new physics manager 1110. Thephysics manager application also at this time processes any physicsresource requests issued by the newly participating physics manager.

One possible data format for the physics resource request might includerequest type, database requirements, entity count, etc. The physicsmanager application would then communicate over the physics farmcommunications backbone 1140 to the newly participating physics manager1110 the results of the physics resource request (i.e., assignresources). For example, in assigning resources the physics managerapplication might assign provide the newly participating physics manageran ID, database, multicast address, etc. Similarly, when the physicsmanager departs or ceases participation, the physics manager application1100 would release the resources previously assigned. A database wouldpreferably include scene data including, e.g., models, entityattributes, etc.

The physics manager application 1100 preferably includes scene managercapabilities for maintaining attributes and states of participatingphysics managers 1110. The participating physics manager thencommunicates with the physics engine (either assigned from the pooledresources or embedded) and provide it with relevant informationregarding the simulation which it just joined; e.g., provide the physicsengine with the relevant scene data and model(s) to load. The physicsengine could load more than a single scene. Scenes could appear asinteracting scenes or they could be independent scenes wherein objectsin one scene do not interact with objects in another scene.

In one embodiment, the physics manager application could manageresources as depicted in FIG. 16, which illustrates a physics managerapplication 1200 communicating with physics farm components over thephysics farm communications backbone 1210. Each physics farm resourceavailable for allocation (i.e., a physics farm component) would includea physics manager 1221-1224 and a physics engine 1231-1234. In theembodiment of FIG. 16, the physics manager application preferablycreates multicast groups and assigns participating physics managers toone or more multicast groups; preferably multicast group assignments areimplemented so as to optimize network efficiency and/or to improvesimulation performance.

Also, in the embodiment shown in FIG. 16, the simulation world isdivided into areas 1241-1244, where each area represents a portion ofthe simulation database loaded into a physics engine. The areas1241-1244 could be assigned by the physics manager application to thephysics farm resources based on any one of a number of factors.Preferably areas are assigned to physics farm resources so as tooptimize computational and network performance of the distributedsimulation. The areas preferably overlap with adjoining areas of thesimulation world so as to support effectively “handing-off”responsibility for simulation entities as they transition from one areaunder the ownership of a particular physics engine to another area underthe ownership of a separate physics engine.

FIG. 17 is an event trace illustrating the dynamic allocation of physicsresources by the physics application manager. A physics manager 1312entering the distributed simulation communicates a request to registeras a resource over the physics farm communications backbone 1330 to thephysics manager application 1300; the registration request includes,e.g., ID, capabilities, etc. of the physics manager. Similarly, otherphysics managers 1322 broadcast a request to register as a resource uponentering the distributed simulation.

The physics manager application 1300 receives the registration requestsover the communications backbone 1330 and issues acknowledgements to therequesting physics managers, confirming registration. Once registered,the physics farm API 1310 can issue a request to the physics manager1312 to initialize the environment and the physics manager 1312 in turnbroadcasts the request over the communications backbone 1330 to thephysics manager application 1300. The physics manager application 1300then allocates the necessary resources to accommodate the request,broadcasting the allocated resources (e.g., ID, capabilities, etc.) overthe communications backbone 1330 to the requesting physics manager 1312.Utilization of the allocated resource can then proceed.

According to another aspect of the invention, the errors, differences,inconsistencies, conflicts, etc., that can exist among the individualparticipating simulation applications are further reduced or eliminatedby avoiding coordinate conversions in the computations that are mostsensitive to slight variations.

As shown in FIG. 18, the current approach using the DISprotocols/network 300 requires that all traffic across the networkutilize the Geocentric Coordinates (GCC). As a result, each simulatorapplication 310, 320, despite the coordinate system being used locallyin the local simulator application, must be converted to GCC 315, 325when broadcast across the network. The receiving simulator applicationsthen have to convert from GCC 315, 325 into whatever local coordinatesystem that particular receiving simulator application was using for itscomputations. This is required even if the receiving simulatorapplication was using the same coordinate system as the remote/sendingsimulator application.

As can be seen by the different positions of the rendered entities 311,312 in each of the simulators 310, 320 in FIG. 18, errors in positioncan result in inconsistent detection or results of collisions.Similarly, differences in line-of-sight (LOS) may exist due toinconsistencies in the different scene data, which impacts “fair fight”,because, as an example, entity 312 has a LOS on entity 311 in simulator320 but in simulator 310, entity 311 does not have a LOS on entity 312.

According to one embodiment of the present invention, conversions areeliminated between physics engine/managers as they would each beoperating in the same global coordinate system. The only point at whichconversion to a simulator's local coordinate system might be required iswhen communicating between the physics engine and the associatedsimulator 335, 345; no conversion is required when a physicsengine/manager communicates 336,346 across the physics farmcommunications backbone 301 with the rest of the physics engines. Insuch an embodiment, physics engines and physics managers are operatingin a common coordinate system thereby eliminating errors due toconversion in the physical representations of entities in local andremote simulators. More specifically, only rendering in the localsimulator can be affected by conversion errors, while the physicscalculations, which are inherently more sensitive to differences due,for example, to the effects of slight errors on collisions or lines ofsight, do not experience this issue because all of the physics enginesuse the same coordinate system.

In the prior art, SAF entities are used to fill in the battle space withfriendly, neutral and opposing forces. SAF forces in prior art systemsare generally controlled as simple kinematical models, primarily toreduce computational loads. As a result, the SAF entities of the priorart are not influenced by physics-based forces, and the result is thatthe SAP entities lack realism. According to the invention, however, anSAF application relies on the physics farm to allow for additionalfunctionality of SAF entities, as, for example, to allow for the SAFforces to influence and be affected by physics based events.

FIG. 20 shows one approach for integrating vehicle man-in-the-loop (MIL)simulators and SAF/computer generated forces (CGF) with the physicsbased system of the invention. The system includes one or more usersimulation stations generally indicated at 510, comprising a simulation511 with a display for the user (not shown), that receives imagery fromrendering system 512 responsive to real time scene data modificationsfrom the physics farm received through physics farm API 513. Physicsfarm API 513 communicates with the physics farm via physics manager 514,physics engine 515 and physics farm communication backbone 530, as hasbeen discussed above.

The physics backbone 530 is linked with SAF computer system 520 whichsupports thereon an SAF application. Using the current military approachemploying the standard DIS protocols, SAF behaviors are bundled withvehicle dynamics where the vehicle dynamics are based upon kinematics(new positions computed for each frame) and the SAF behaviors are basedupon defined echelon hierarchy and standard military doctrine.

The SAF application is divided into two separate applications, the SAFPhysics Manager Application (the native application) 521, and a separateSAF behaviors application 526. Each application has a respectiveinterface with the physics farm via entity management APIs 522 and 527,and Physics Farm APIs 523 and 528. These APIs communicate with arespective physics manager 524 and 529 for each SAF application. Thephysics manager integrates the SAF requests from the SAF applicationswith a physics engine 525, and provides feedback to the SAF physicsmanager application 521 or the SAF behaviors application.

The requests of the SAF applications result in creation or movement ofphysics-based entities in the virtual world, and those entities canaffect users or the simulated environment based on the physics basedinteraction of entities discussed above, e.g., SAF forces may be able toshoot, to put weight on objects and cause them to fall, or to engage inany kind of activity that produces force on the entities in the virtualworld. In addition, the SAF entities can be the recipients of forcesthat affect them, and that alter their conduct based on their strategicrules.

According to the preferred embodiment, behaviors are separated fromvehicle dynamics, requiring position data feedback to the SAF behaviorsapplication only when the SAF directed request is complete, the requestis interrupted or a collision occurs. As can be seen, the proposedentity management protocol and entity management API permit the SAFbehaviors to be separated from vehicle modeling. Additionally, theentity management protocol provides mechanisms for SAF behaviors todirect the movement of physics-based vehicle models. The SAF physicsmanager is configured to integrate SAF requests from the SAF applicationwith the physics engine and for providing feedback/callback routines.

The data flow of the physics interface with the SAF application isschematically illustrated in FIG. 21. A simple message protocol is usedas the interface between the physics engine interface and SAPapplication, and the messages are transmitted via multicast socket.

During the creation of SAF entities, the SAF application sends out dataindicative of the initial position, orientation, and entity type to thephysics engine interface for those entities that will be modeled in thephysics engine. Based on the entity data received, the physics engineinterface then creates a physics model in the physics engine for eachSAF entity, so that it begins to exist in the simulated environment.

The SAF application periodically sends an update to the physics engineinterface to update the physics models of the SAF entities.

When the physics models of the SAF entities interact with other objectsin the virtual environment, creating a collision in the physics engine,any position, orientation, and velocity changes to the physics models ofthe SAF entity involved is sent back to the SAF application, whichupdates its corresponding entity based on the updated position,orientation, and velocity.

Behavior of the SAF entities in the simulated environment follows thebehavior of the corresponding entity defined in the SAF application andprocessed therein. Complex SAF behaviors resulting in realisticinteractions with the physics based simulation are made possible by thisinterface.

EXAMPLE

FIG. 22 is a simplified block diagram of a demonstrator used to provethe feasibility of applying physics-based modeling for applications thatcurrently rely only on kinematics modeling. Additionally, physics-basedmodeling was used to incorporate dynamic “cultural features,” which, ifprovided for at all, have typically been modeled statically.

The demonstrator provided higher fidelity models, increased quantitiesof vehicles and new dynamic cultural features. In addition to theinclusion of vehicle models, dynamic cultural features were incorporatedinto the simulation by loading cultural feature models into the physicsengine. This allowed for interactions between the cultural features andvehicles in the simulation. For example, using the physics-basedapproach, a vehicle was now able to push a barrel out of the way, withboth the barrel and the vehicle responding in a realistic manner to theforces generated by the interaction.

The key components in the example shown in FIG. 22 included a physicsengine 71, a Semi-Automated Forces (SAF) application supported in anAviation Combined Arms Tactical Trainer (AVCATT) 73, a dynamicenvironmental model 75 for cultural features, intelligent behaviors 77and an NGT bridge 79.

The physics engine 71 used in the demonstrator of FIG. 22 was acommercially available software product that provides a softwaredevelopment kit (SDK) and algorithms for modeling a physics-basedvirtual world. Objects in the physics engine 71 can be created,attributed and registered with the physics engine. The physics engine'salgorithms can either be implemented in software or hardware via a PPU.

The legacy based AVCATT SAF application 73 was adapted to work in thephysics-based proof of concept demonstrator by first removing theexisting dynamic models from the behaviors that direct the movement ofthe vehicles. In particular, the SAF helicopter and wheeled vehiclemodels were replaced with helicopter and wheeled vehicle modelsdeveloped using the physics engine SDK and registered/executed on thephysics engine 71.

As adapted, the SAF behavioral commands (e.g., “move”) directed to thevehicle dynamics are sent to the dynamic models hosted on the physicsengine 71 and the results or progress are/is sent back to the SAFbehavioral logic to continue reasoning in accordance with the behavioralartificial intelligence implemented in the SAF application 73. Inaccordance with a preferred embodiment of the present invention, the SAFapplication 73 sent the physics engine 71 the type of model to becreated (e.g., helicopter, wheeled vehicle, etc.) along with itsrequired attributes such as mass, inertias, dimensions, etc.

Once the model is instantiated and registered with the physics engine 71the SAF application 73 directs the physics-based model behavior (e.g.,directs it to move). Preferably, position updates are periodically sentfrom the physics engine 71 to the SAF application 73. In order to enablethe SAF application 73 to direct requests and receive results from thephysics engine 71 hosting the physics-based objects/entities (e.g.,helicopter, wheeled based vehicle, etc.), the demonstrator utilizedcommunications protocols, message formats and APIs (developed to processSAF application 73 requests and communicate results) in accordance witha preferred embodiment of the present invention, the details of which ordiscussed in greater detail below.

In addition to the physics-based dynamic models (i.e., helicopter, wheelbased vehicle, etc.) that are directed by the SAF application 73,cultural feature objects (e.g., dynamic buildings) can be created andregistered on the physics engine 71 by the dynamic environmental model75. Advantageously, the cultural feature objects, e.g., barrels, crates,boxes, barricades, and the like, can be seamlessly integrated into theenvironment and will interact with other physics-based objects, such ashelicopters, ground vehicles, life forms, etc. The dynamic buildings maybe assembled from objects that interact with the physics environmentindividually and can break apart based on forces being applied asopposed to scripted events. The forces can be generated by explosivedetonations, impacts, from other dynamic objects, etc.

Life form behaviors have traditionally consisted of virtualrepresentations of people participating in a simulation. The intelligentbehaviors component 77 can model life forms using a physics-basedapproach so that the life forms may be treated like other dynamicobjects, such as air or ground vehicles. In accordance with a preferredembodiment of the present invention, feedback mechanism would berequired to communicate requests and to receive results of the effectsof forces from the physics engine 71 that computes the physics basedinteractions/operations.

In order to interface the physics engine with other more traditionalsimulations based on the DIS protocols, an NGT bridge 79, as discussedabove, was employed. The NGT bridge 79 enabled entities modeled bysimulations external to the physics engine application to beincorporated/injected into the physics-based environment; these externalsimulations are preferably injected as kinematics objects since theirmotion/behavior is governed by external forces. Accordingly, accordingto a preferred embodiment of the present invention, entities nativelymodeled by the physics engine 71 would be distributed to the externalsimulations using the DIS protocols.

Synchronizer System

The system as described so far provides some substantial advantages.However, three potential problems should be addressed when using physicscoprocessors to simulate objects in a distributed environment. One isthe problem of network latency. A second problem arises from the factthat multiple physics engines that have been provided with identicalinitial conditions will, over time, inevitably provide solutions thatdiverge from each other. The third potential issue is that objects inthe physics environment will interact with other objects in the virtualenvironment and may move in a manner not intended by the controllingsimulation. For example, a computer controlled vehicle may be influencedor damaged by a collision created by latency and move in an erraticunintended path in certain of the simulations affected by that latency.

These problems can be addressed using periodic positional updates thatallow the various simulator systems to synch up and match theirrespective versions of the scene data of the virtual world. Althoughthis is workable, the periodic positional updates consume networkbandwidth and computer processing resources in proportion to the numberof simulated objects. This bandwidth demand makes it increasinglydifficult to perform distributed simulation exercises that includethousands of objects, which means that it is less practical to attemptexercises with the tens of thousands of objects desired for certainvirtual environments, such as dense urban scenarios.

Referring to FIG. 23, a system of master and slave synchronizers orcontrollers is provided that may be used with the physics based systemof the invention to enhance the synchronization of the system of theinvention with less of a bandwidth load. This system implements anapproach to distributing simulated objects across networks simulationsthat reduces the impact of network latency and also allows for a muchlarger number of objects in the virtual environment without aconcomitant consumption of network bandwidth and computationalresources. The system also preferably relies on physics coprocessorsand/or the physics farm discussed above for the associated control ofobjects simulated in the virtual environment of the simulation, as willbe described below.

The system comprises a plurality of simulation stations 703, which aresimulation systems similar to systems 3 as discussed above with respectto FIG. 1. The system also includes one or more client entity or objectsimulation computer systems 705, which are computer systems such as SAFcontrolling computer systems 9 discussed above. The simulation stations703 and simulation computer systems 705 are connected via a network 707,which is preferably the network 5 of the system of FIG. 1, e.g., theInternet or a WAN. Only two simulation stations 703 and one simulationcomputer system 705 are shown in FIG. 23 for illustration purposes, butthe system preferably encompasses a substantial number of each type ofstation 703 or 705 to support a number of trainees and also a relativelylarge number of computer-controlled client objects to enhance realism ofthe simulation.

Simulation Computer Systems

The simulation computer systems 705 are systems that control thebehavior of client objects in the virtual world of the simulation,particularly SAF entities (such as enemy forces, vehicles or weaponsystems), the simulation of which is discussed above, and illustratedin, e.g., FIG. 14. Essentially, each simulation computer system 705 is acomputer system with a connection or access to a data storage devicestoring a respective version of the simulation scene data. A behaviorsoftware application 709 runs on the computer system, and determines,based at least in part on the scene data, the behavior of the clientobjects, e.g., SAFs or intelligent entities assigned to that system 705.

Generally, the behavior is movement of the simulated client entity orobject, but may include any sort of self-actuated or reactive behavior,as discussed above, and is well known in the simulation art. Forexample, the behavior can include behaviors such as, e.g., a deviceexploding (usually expressed as extremely high speed movement of thediscrete parts of the exploding object).

Within the behaviors appropriate to the client object, the behavior iscalculated by the behavior application software so as to yield behaviordata defining the behavior. Because the behavior data is usually datadefining an intended projected path or movement of the client object,the behavior data for most objects simulated, especially vehicles orsimulated humans, constitutes client route data.

In the preferred embodiment, the client route data includes at least adesired speed, one or more route points defining the route that theclient object is to pass through in the movement, a desired ETA at theend of a route, and an append flag.

The route points are preferably two- or three-dimensionally expressedpoints in a client coordinate system. Where the client object is onethat moves along the ground in the virtual simulation, such as avehicle, an animal or a walking person, the coordinates can betwo-dimensional, because the client object moves in two dimensions, withthe vertical movement of the client controlled by the height of theterrain it is situated on. Where the client object is one that is ableto initiate its own movement in three dimensions, such as a submarine, afish, a helicopter or a bird, then the route is expressed in threedimensionally defined points.

The ETA is expressed using a system-wide time clock system, such as thesystem-wide network time protocol (NTP). From the standpoint of themaster synchronizer, the ETA defines a future point in time by which theclient object should reach the point at the end of the defined route. Inaddition, data defining an ETA for intermediate points along the definedroute may also be provided in the client route data.

The append flag is a flag of data that is set to indicate whether theroute being requested is to be appended to the existing route for theclient object, or if the new route is intended to simply replace thecurrent route for the client object.

The Master Synchronizer

The behavior or client route data from the behavior application passesor is otherwise communicated to master synchronizer or controllerapplication software 711, which preferably also runs on simulationcomputer system 705. The master synchronizer 711 receives the behaviordata and based thereon handles the creation of the virtual components ofthe simulated client object within the physics engines, as well as thelow-level control and synchronization of the object within thedistributed environment.

Each client object simulated within the physics environment has one, andonly one, master synchronizer 711 that communicates with the applicationthat is simulating the client object. Client behavior application 709provides behavior or client route data to the associated mastersynchronizer 711. Distributed control and synchronization of the cliententity or object over the system is then accomplished usingsynchronizers in a master/slave relationship as shown in FIGS. 23 and24.

The synchronizer route data for a client object is derived from thebehavior or client route data received by the master synchronizer 711,and is configured for distribution over the network. Generally, thesynchronizer route data typically includes data defining a goal and astart time. The data defining the goal may be a destination point, or itmay be data that defines a route and a desired speed that the clientobject is to proceed along the route, or it may be another combinationof data defining some time-dependent locations and/or orientations forthe client object. For example, the synchronizer route data may containdata defining a route and an ETA at an endpoint of the route, i.e., apoint in time that is at least a short time in the future from thestandpoint of the master synchronizer at which it is desired that theclient object reach the endpoint of the route.

The synchronizer route data according to the preferred embodiment alsoincludes an append flag indicating, on or off, whether the routetransmitted is to be appended to the currently in-place route, or if itshould replace the current route.

Where the goal is defined using one or more route points, the points arepreferably are defined in two or three dimensions depending on whetherthe object is capable of flight or simply surface movement, and arepreferably x,y,z or x,y points in the internal coordinate system of thephysics environment data accessed by the physics engines. In addition,duplicate points and small route segments in the behavior data, both ofwhich represent unnecessarily transmitted data, are removed.

While there is one and only one master synchronizer for each clientobject in the virtual world of the simulation, the master synchronizer711 spawns, in each simulator station computer system 703 to which thesimulated entity or client object is relevant, creation of a respectiveslave synchronizer 713 for that station's physics environment. As aresult, for each client object relevant to the simulator station 703,there is a respective instance of the slave synchronizer program 713that controls the client object. Master synchronizer 711 transmitssynchronizer behavior or route data for the associated client objectover the network 707 to slave synchronizers 713 at the variousdistributed simulation stations 703.

The Slave Synchronizers

At each simulation station 703, the slave synchronizer 713 receives thesynchronization route data, and interacts with the physics engine 715 ofthat station 703 to cause the client object to follow the directed routein the stored physics-based environment so that, in the associatedsimulator system 703, the client object meets the goal or goalsspecified by the synchronization route data within the specified time ortime-dependent constraints. The slave synchronizer 713 is preferably asoftware program running on the computer system of the simulationstation 703, and one copy of the program runs on each simulation stationin which the client object is relevant to the local simulation. Thephysics engines 715 of the simulation stations 703 are preferablyconnected as parts of the physics backbone discussed above, exchangingphysics data over the system as discussed above, with physics managementsoftware running on the simulation system assigning the responsibilityfor physics of the objects as has been set out previously herein.

The local slave synchronizer program 713 receives the synchronizer dataas input from a link to the network, and controls movement of the clientobject in the local physics environment simulation by passing ortransmitting to the physics engine 715 physics data reflectingappropriate forces to be applied to the client object or to itscomponent virtual objects, e.g., the drive wheels of a vehicle, or theparts of an exploding device, that will cause the client object to moveto reach the goal at the appropriate time. The physics engine 715 inturn calculates the solution of the physical forces introduced thatreflects the consequences under the internally-define physical laws ofthe physics engine 715 of the forces from the slave synchronizer (andany other synchronizers in the station 703), meaning changes in locationor orientation or velocity of one or more objects in the virtual scene.The physics engine modifies the physics data stored therewith, andaspects of the physics data, i.e., the new locations of objects in thevirtual scene, are passed or moved to be incorporated in the scene data717 stored at the station 703, similarly to the procedure describedabove. The image generator 719 accesses the scene data or data derivedrenders imagery from the scene data which is then displayed to the userat the simulation station 703 at whatever display device 721 isassociated with the station 703. In addition, the slave synchronizer 713accesses the scene data to determine the updated location of the clientobject after the physics data transmitted to the physics engine isimplemented in the virtual environment, as will be expanded upon below.

A key aspect of the synchronizer system is that the local slavesynchronizer 713 has direct access to the client object as it is modeledwithin the local physics environment and can therefore providetightly-coupled, high-frequency control of the physical model used atthe simulation station 203. Due to this local control, the simulatingapplication 709 does not need to attempt to provide this high frequencycontrol across the network.

Master synchronizer 711 in this illustration is part of a server 709without an image generator or viewer. However, the master synchronizermay be on a system such as stations 703 that has a physics engine andimage generator. In that case, the master synchronizer transmits forcedata to the physics engine, which calculates the effect of the forcedata on objects in the scene according to its internal physical rules,resulting in a modification of its internal physics data defining theworld. The changes to the physics data of the physics engine aretransmitted to the local computer system of the station 703 and thescene data used to generate the imagery on the simulator display isupdated, and the local image generator displays the modified virtualworld based on the physics data.

In addition, as will be expanded on below, the client computer systemthat is running the slave synchronizer is configured to, under theappropriate conditions, to load appropriate software and become the hostcomputer that runs the master synchronizer for a virtual object.

Operation of the Synchronizer System

The behavior applications 709 of the system determine the timing androute of movement for the associated client object according to the typeof object it is.

Generally, there are two classes of simulated client objects andassociated synchronizer algorithms that are the basis of mostsynchronizer implementations: (1) simulated objects that exhibitbehavior, and (2) simulated objects that do not exhibit behavior.Examples of objects that exhibit behavior are vehicles and personnel.These objects generally conduct themselves with a purpose and toward theachievement of a goal. Examples of objects that do not exhibit behaviorare buildings or shipping crates. Objects such as those tend to remainin their initial state unless acted upon by some external force.

Synchronizers for objects that exhibit behavior implement algorithmswith two characteristics. First, a deterministic algorithm defines theimmediate goal of the object behavior. Second, a closed loop controlalgorithm that understands the nature of the physical model drives ittoward the immediate goal. This provides the advantage that once thegoals, relevant time parameters for the goal, and a start time, havebeen communicated between master and slave synchronizers, no furthernetwork communication is necessary for the objects to achieve theirgoals and to remain in synchronization. Network latency ceases to be anissue, because the deterministic goals allow the objects to move withoutconstant control over a latency-affected communications link.

The closed loop control solves the possible issue of divergent physicsengine solutions at different simulation stations that could result ifleft to run under open loop control. The closed loop control alsoaddresses the possibility that an object in the local physicsenvironment may not follow the path that the controlling simulationintended. The closed loop control quickly corrects any perturbation fromthe intended path due to interactions with other objects within thelocal physics environment.

Generally expressed, the slave synchronizer operates according to aclosed loop to cause the client object to achieve the goal at thecorrect time. In the closed loop, the slayer synchronizer receives theroute data and derives from that data a current set of physics data forthe client object configured to cause the client object to advancetoward the goal, and that data is passed to the physics engine, whichimplements the physics data for the client object in the virtual physicsenvironment data according to the physics rules, resulting in change ofthe scene data. The scene data is also accessed by the slavesynchronizer, which calculates a new set of physics data based on theupdated position of the client object in the scene data and on the mostrecent route data that has been received from the master synchronizer,and new physics data is derived for the client object to cause it toreach the currently specified goal at the currently specified time. Thisnew physics data is passed to the physics engine, and the scene data isagain updated. The cycle is completed within the duty cycle for theimage generator in refreshing the rendered video images displayed to theuser at the simulation station, e.g., 1 Hz, and is preferablysubstantially faster.

The slave synchronizer 713 at station 1 performs a process in which adetermination is made whether there is a conflict in the physics-basedenvironment preventing or delaying the client object moving according tothe desired synchronizer route data, and, where there is such aconflict, the synchronizer adjusts its physics data for the clientobject to compensate for the obstruction or delay to try to cause theclient object to reach the goal point at the appropriate time.

According to the preferred embodiment, at each cycle of the slavesynchronizer, a calculation derives the appropriate point in the virtualworld where the client object should be if proceeding along theidentified route at the prescribed rate and with the directed startingtime. This is compared with the current actual virtual location of theobject according to the scene data. The difference is used to computeappropriate physics data that is configured to cause the client objectto accelerate (or decelerate if it is moving too fast for the rate) andto get the client object on to the route if it has gone off theprescribed route.

Compensation for Delay

Operation of the synchronizer system of the preferred embodiment isillustrated by an example shown in FIGS. 29 and 30.

FIG. 29 shows a desired route for the client object vehicle. Thesynchronizer data for the client vehicle 725 includes the definition ofthe destination point E, and the intervening points B, C and D throughwhich the vehicle 725 is directed to pass, as well as timing data thatpreferably includes data defining a rate for the route or arrival timesfor at least the end point (t₄), starting at time t₀. Based on thespecified rate, the slave synchronizer calculates that vehicle 725 is tobe at points B, C, D and E at times t₁, t₂, t₃ and t₄, respectively,and/or optionally arrival times t₁, t₂, t₃ and t₄ for points B, C, D andE on the route may be present in the synchronization data. Thesynchronization data includes at a minimum the time and place data onwhich the slave synchronizer can derive that the vehicle 725 is toarrive at point E at t₄ after passing through the intermediate routepoints defined, and from that derive the appropriate timing of theintermediate arrivals.

FIG. 30 shows a scenario that may arise in the physics environment, andthe consequent local scene data, at a given simulation station 703remote from the behavior application computer system 705. In the localphysics environment, an external force F interacts via the local physicsengine 715 at the simulation station 703, delaying the vehicle 725 inits timely progress along the route.

At time t₁, the local slave synchronizer 713 determines that the clientvehicle has been delayed and has not yet reached point B, as directed bythe synchronization data. The local slave synchronizer 713 thereforeacts on the physics engine of the station 703 to accelerate the clientvehicle 725, i.e., increasing the forces moving the vehicle forward orincreasing torque on the drive wheels of the vehicle, after t₁ so thatthe vehicle can make up for the delay.

At time t₂, the vehicle in the example is not yet at point C where itshould be, so further acceleration (up to an appropriate limit for thesimulated vehicle) is applied to the vehicle 725. At time t₃, however,the vehicle being simulated has not yet, within the physics rulesconstraints of the physics eng, been able to reach the scheduled pointD. The slave synchronizer applies further acceleration as permitted bythe simulated vehicle constraints stored therein, and gets the rate ofthe vehicle above the appointed rate, so that the vehicle reaches theendpoint of the route E at the proper point in time, t₄. As a result ofthese accelerations, the vehicle proceeds at a higher rate along theroute segments CD and DE than originally scheduled by thesynchronization data illustrated in FIG. 29.

Vehicle Speed and Steering Control Loops

Where the client object is a vehicle, the slave synchronizer applies thecontrol loop and method diagrammatically expressed in FIGS. 31 and 32 tocontrol its movement. This illustration is expressed in analog format,but the preferred embodiment implements to the logic of the diagramusing software recorded on a computer-accessible data storage medium andoperating on a digital computer at the simulation station, whichcomputer may share in other activities, e.g., providing thecommunication link of the physics engine with the physics backbone, orthe local portion of physics management of the physics farm.

In this specific vehicle application, the slave synchronizer receives orcalculates for a route two values: the distance from the next goal point(e.g., as in FIG. 29, the appropriate point on the route for the currenttime) indicated at line 733 and the rate at which the vehicle is toproceed to reach that endpoint at the point in time prescribed by thebehavior application, indicated at line 735. The distance to goal isthen scaled by control gain or constant calculation K₁ to yield a datavalue scaled to the data value of the desired speed, which is actually adistance value for a unit of time, such as the duty cycle. These twovalues are compared at comparator 736 to yield a difference value, i.e.,a speed value indicative of the rate needed to reach the goal point atthe appropriate time. This speed value is further scaled by control gainor constant calculation K₂ to yield a commanded speed 737. Thiscommanded speed 737 is compared by comparator 739 with the currentvehicle speed (line 738) to yield, after control gain or constantcalculation K₃, a difference value corresponding to a commandedacceleration 740. Commanded acceleration 740 is compared with currentvehicle acceleration 741 at comparator 742 to yield a data value that isintegrated in step 743 to produce a data value indicating drive wheeltorque 744 to be applied to the drive wheel or wheels of the vehicle,which are physically modeled in the physics environment data asindependent parts of the vehicle. The drive wheel torque data value isprocessed by a physics drive wheel model 745, which is configured todetermine the specific data values to be defined for application via thephysics engine to the physics of the client vehicle, i.e., the vehiclespeed 746 and the vehicle acceleration 747. These data values are thenimplemented in the physics engine so that the client vehicle acceleratesor decelerates to reach the indicated goal point at the correct time.The data values are also looped back and the desired acceleration isre-calculated for the next cycle of the loop.

To keep the vehicle on the defined route of the synchronization data, oron a revised route created by the slave synchronizer to avoid anobstacle on the route, the steering direction of the client vehicle isalso controlled in the slave synchronizer by a closed loop. FIG. 32illustrates the operation of this aspect of the slave synchronizer,which is also implemented in the preferred embodiment in softwarerecorded on a computer accessible data storage medium and running on adigital computer system.

In the process disclosed, data defining the route segment heading 751,meaning data defining the direction that the client vehicle is to followaccording to the route defined by the associated behavior application,is combined with a data value 752 derived from a control gain of adetermination of the distance 753 of the client vehicle perpendicularlyfrom the route by comparator 754, to yield a data value corresponding toa desired heading 755 for the vehicle to follow the route. This desiredheading 755 is compared with the current vehicle heading 756 atcomparator 757 to yield the raw steering command 758. This data value758 is limited to a permitted range of steering directions to yield adata value corresponding to a final steering command 759, which isprocessed by a physics steering model 760. The physics steering model760 generates a vehicle heading data value 761, which is implemented inthe physics engine. The implementation may correspond, as illustrated,to turning the independently modeled steering wheels of the clientvehicle, with resulting movement of the vehicle being guided therebyaccording to the laws of the physics environment. The heading is alsolooped back and the desired heading is re-calculated, yielding a newsteering command for the next cycle of the loop.

Analogous control loop processes, i.e., speed and direction, can be usedto control movement of virtually any client entity in the virtualenvironment of the simulation.

Life Form Movement Control Loop

Life forms, such as human beings, are modeled in the physics world as acolumn supported on a wheel, for purposes of movement. The column isconfigured as always vertical, so there is no balance involved in themovement of the life form. The scene data associated with the life form,however, is a more realistic detailed image of the person or animal thatreflects its position as defined by the physics environments data buthas the other visible attributes of the life form, e.g., head, legs,arms, clothing, fur, etc. The speed of movement of the life form iscontrolled as above for a wheeled vehicle, with the drive wheel beingthe wheel of the physics model. The direction of movement of the lifeform is controlled by applying rotational speed and acceleration aboutthe vertical axis.

The method illustrated in FIG. 33, preferably implemented in software asdiscussed above, uses a joystick input 771 for the life form, as when aninstructor controls a SAF movement in the virtual world. The joystickinput is passed through control gain K₁ to yield a desired angularvelocity 773, which is compared with the actual angular velocity value774 of the life form from the last command to the physics engine toyield a difference value that is passed through control gain K₂, toyield the desired angular acceleration 775. This desired angularacceleration 775 is compared with the actual angular acceleration value776 of the life form from the last command to the physics engine and isintegrated to yield a value representing body torque 777 to be appliedto the life form model column. This body torque value 777 is used by aphysics life form model 779 to yield an angular velocity value and anangular acceleration value 781, which are sent to the physics engine andimplemented in the scene data. These values are also returned and usedfor comparison as actual values 774 and 776.

Where the control of the life form is by a software application runningon a simulation computer system, and not by human input, as by ajoystick, a similar process of control loop is employed, but where adesired angular rotation and velocity are calculated by the slavesynchronizer using the synchronizer route data. A comparison of thedesired heading and the desired rate are made as with the vehicle above,and are implemented similarly to keep the life form on route and onschedule per the synchronizer route data.

Human input controlling entities in the physics based environment isnormally accomplished by input from an instructor station equipped withan input such as a joystick, and a display showing the virtualenvironment to the instructor. Instructor control of virtual entities isnot limited to SAF entities or life forms. The instructor can assumecontrol of any virtual entity in simulation where that entity is capableof movement. Furthermore, the compute system local to the instructorwill become the system that supports and hosts the master synchronizerfor the object until the instructor relinquishes control.

Other Operational Aspects

A simple example of one of the potential problems that can beencountered with the physics engine system, e.g., differingphysics-based conflicts among the distributed simulation systems, isillustrated in FIGS. 25 and 26.

Referring to FIG. 25, the exemplary virtual world includes two objects:a SAP vehicle 725 that, according to its associated behaviorapplication, is instructed to proceed immediately to point A, and avirtual object 727 that is being propelled by some force to move topoint B. In the example here shown, both of these movements are to occurin roughly the same time window in the overall system simulation.However, if the instructions to objects 725 and 727 are delayeddifferently at different simulation stations due to a latency Δ, thenthe results may be divergent.

As best seen in FIG. 26, according to this illustration, in a systemwithout the synchronization here described, at station 1, the vehicle727 a in station 1 leaves at t₀+Δ, while the block 727 a starts movingat t₀, reaching an obstructing or blocking position at t₀+1Δ. Thevehicle 725 a meets the block 727 a at t₀+3Δ, well after the block hasarrived, and is prevented by the physics rules from proceeding to pointA as directed, because of a conflict with the block 727 a.

On the other hand, at station 2, the latency delays block 727 b, but notvehicle 725 a. Vehicle 725 b starts out on its route at t₀, but atstation 2, due to latency, block 727 b does not start to move untilt₀+1Δ, only reaching its position blocking the route at t₀+2Δ. By thattime of t₀+2Δ, however, the vehicle 725 b has already passed thelocation of the obstruction, and proceeds to point A without obstructionas directed.

As a result, the scene data defining the two virtual worlds of station 1and station 2 reflect two divergent solutions, one in which the vehiclegets to A, and the other in which the vehicle is obstructed by the blockand never reaches point A.

Where the master-slave synchronizer system is applied to the systems ofthe illustration in FIG. 26, this problem of divergent solutions isavoided. The master synchronizer for the vehicle 725 transmitssynchronizer data to the slave synchronizers at stations 1 and 2 thatdirects the vehicle 725 to proceed to point A, and that includes datadefining a start time t₀ for the movement, defining the route bycoordinates of at least point A, defining a rate of travel along theroute certain rate so that vehicle 725 arrives at point A by a certainpoint in time, e.g., t₀+x, and, optionally, defining the specificdesired ETA t₀+x at point A.

At station 2, the slave synchronizer transmits or passes physics datafor the vehicle to the physics engine at that station that causes thephysics engine to move the vehicle, or its constituent virtual parts, inthe physics environment data at an appropriate rate and reach A at theappropriate time. Due to the latency delay in the start of the movementof block 727, by the time that the block 727 arrives at an obstructingposition, the vehicle 725 has already passed the obstructed part of theroute to A. The vehicle proceeds to point A by t₀+x at the specifiedrate along the route. Because there is no conflict with the block, themovement is essentially the same as the movement of the vehicle 725 atstation 2 in FIG. 26.

At station 1, in contrast, due to latency in the communication of thedesired vehicle movement, the vehicle 725 a starts moving, but, due tothe absence of latency for the block at station 1, the block 727 aarrives quickly enough to block the route of the vehicle 725 a to A,resulting in the conflict as illustrated in FIG. 26. This is implementedin the scene data and rendered by the image generator, and the user atstation 1 sees the contact of the vehicle with the obstruction on hissimulator display if it is part of his appropriate view of the virtualworld.

In the next cycle of the local slave synchronizer 713 of station 1, thelocal slave synchronizer 713 compares the obstructed position of thevehicle 725 a (against the block 727 a) to the position in which thevehicle 725 a is scheduled to be in, based on the start time and therate data values from the synchronizer data (on the other side of theobstruction 727 a). Based on the determination that the vehicle isdelayed, the closed loop control of the synchronizer 713 transmitsphysics data to the physics engine to move faster. This will result inan increase of force from the vehicle pushing in the physics environmenton the obstruction 727 a.

If the vehicle is able to generate enough force, pursuant to its modelin the local synchronizer 713, it will eventually push the block 727 aout of the way. The result of the pushing is resolved by the physics ofthe conflict, e.g., by how heavy the virtual object 727 a is supposed tobe in the simulation. Ultimately, there are three possible outcomes: theblock is pushed out of the way, there is a deflection of the vehiclefrom the route, or the vehicle is obstructed.

Where the block is simply pushed out of the way, to the extent that theblock has delayed the vehicle, the vehicle will push the block out ofthe route, and then accelerate to reach the goal point A by theprescribed time.

Where the block is partially pushed out of the way, but the vehicle isdeflected from the route, a similar result will follow: the steeringcontrol loop of the synchronizer will guide the vehicle back to theroute, and the vehicle will accelerate to the goal point A to get therein time.

In either of these two cases, the physics farm will subsequently resolvethe position of the blocks 727 a and 727 b (and of the block in anyother simulation stations across the system) to conform to a singleposition in the virtual physics environment and the virtual worlddefined by the scene data by a positional update over the entiresimulation system. As a result, after a short period, the solutions ofstations 1 and 2 will not be divergent if the vehicle can push the block727 a out of the way in its route.

In contrast, where the vehicle is obstructed and unable to push throughthe block, a different remedial action is taken by the synchronizer toensure that the vehicle arrives at the goal point at the appropriatetime. The inability of the vehicle 727 a as modeled to generate enoughforce to move the obstruction 727 a out of the way, will result in adelay that will continue for a period of time with the vehicleeffectively obstructed by the block 727 a. The synchronizer in thissituation determines that a delay has been created that is not remediedby acceleration of the client object.

The determination of obstruction may be made in a number of ways. Forone, when the client object should be moving, but its rate of movementis zero or below a predetermined threshold value, for a single cycle orfor a time period that is longer than a predetermined time period or athreshold number of cycles of the synchronizer loop, it is concludedthat the client object is obstructed. Alternatively, the conflict may bedetermined based purely on the contact event and an analysis thereof.

In a further alternate embodiment, the determination of obstruction maybe accomplished by extrapolating the route of the vehicle ahead by one,two or more duty cycles of the slave synchronizer and noting if there isany obstruction thereof in the physics engine without permitting thescene data to be modified until the actual intended movement isdetermined.

Whatever the way the conflict is detected, responsive to thedetermination of the obstruction of the route, synchronizer 713 takesremedial action to ensure that the vehicle 725 a reaches point A intime.

One possible type of remedial action is accomplished by the synchronizerby causing the client object to move in reverse for a short period,e.g., a duty cycle of the synchronizer, to disengage from theobstruction, and then extrapolating a new path of movement for thevehicle that circumvents the obstruction. This new path, identified byreference character 730, allows for the vehicle to proceed to point A,circumventing the blockage of the originally intended path 731, and thenrejoins the original route 731 after the obstruction, as illustrated atFIG. 27. The new path is determined based on the scene data defining theobject that obstructed the route and simply extending a route around theobject that meets the original route on the other side of theobstructing object by any route planning program known to those of skillin the art.

A similar action may be taken where the obstruction is detected byprojection of the client object route without moving the client objectin reverse to disengage from the obstruction. This is done by the slavesynchronizer itself, as opposed to determining a conflict using thephysics engine. The slave synchronizer accesses data stored at itsclient system that defines the locations of objects in the virtualenvironment, e.g., the local physics data or object state data, anddetermines if any obstructions are in the path of the client object.Responsive to a determination of the presence of an obstruction on theroute, a new route is prepared that avoids the obstruction and allowsthe client object to reach its goal point in time. The route is preparedusing any of a number of route-making algorithms, as are well known inthe art.

As another alternative, or as a second option where an unobstructedalternate route, such as that of FIG. 27, cannot be found by the slavesynchronizer, the synchronizer 713 may direct the physics engine tosuspend the physical rules that create the obstructing conflict. As aresult, the obstructing effect of the intervening object, e.g., block727, disappears, which allows the vehicle 725 a to proceed to point A asif there were no block of the route. This is illustrated in FIG. 28.Vehicle 725 a simply proceeds to point A without regard to the presenceof block 727 a. In the image generator output, which is based on scenedata derived frame by frame from the physics data, including theposition of the vehicle, the result will appear to violate the laws ofphysics somewhat, in that the vehicle goes straight through theobstruction, but this brief appearance of unrealistic imagery ispreferable to the greater problem of diverging virtual environments.This suspension of the physics rules may be applied only when certainprerequisite conditions apply, e.g., that a conflict exists, and alsothat there is no route that can be calculated around the conflict incompliance with the physics rules of the physical objects in the localversion of the virtual physical environment.

The suspension of the physical rules is preferably accomplished by theslave synchronizer interacting with the physics engine to make thephysics engine act as the obstructing object was not there, but with therestriction that this temporary disappearance of the obstructing objectfrom the physics environment data does not affect the its consequentpresence in the simulation imagery generated at the simulation station.

For objects that do not exhibit behavior, the closed loop control aspectof the synchronizer system is ordinarily not necessary but is stillavailable. In such an application of the synchronizer system to anon-behaving object is moved due, e.g., to the application of a forcefrom the physics environment, each physics engine of the physics farmindependently solves for the motion of the non-behaving object. When thenon-behaving object has come to rest, or has achieved some intermediateposition, if the master-slave synchronizer system will cause the slavesynchronizer 713 to synchronize the state of the object across thenetwork with a single positional update from the master synchronizer 711for the non-behaving object to each of its slave synchronizers 713.

FIG. 34 shows a flowchart illustrating the operation of the slavesynchronizer according to the above process of the preferred embodiment.First, the software determines whether the client object is obstructed.If not, then the synchronizer performs the control loop adjustments ofthe client object speed and direction as discussed above, and then theloop repeats in another cycle. If there is an obstruction detected, thenthe synchronizer tries to determine a new route around the obstructionusing the physics environment and/or scene data. If it is possible tocalculate such a new route, then the synchronizer causes the clientobject to backup to disengage from the obstruction, and then proceedalong the new route using the new route to guide the client object usingthe client model acceleration and steering control loop. If thesynchronizer is unable to calculate a route around the obstruction, thenthe synchronizer suspends the physics of the conflicting object orobjects that are blocking the client object, so that the client objectcan proceed through the interfering objects without conflict orcomplying, for a brief period, the physics rules of the physics enginein regard to those obstructions.

Details of Synchronizer System Architecture

The details of the architecture of the system of the preferredembodiment employing the above described synchronizer system isillustrated in FIG. 35 et seq.

As best seen in FIG. 35, the system comprises client side applicationsand server side applications each connected a physics-based environmentgenerator (PBEG) infrastructure, which is distributed over the systemusing a physics communications backbone similar to the network discussedabove. The client side applications include the simulators 783 of thesystem, the image generators 784 associated with simulator stations,environment applications 785, applications controlling intelligentbehaviors 786, as well as distributed interactive simulations 787connected via a network bridge, all of which interact with the PBEGinfrastructure, providing data to it and/or receiving data from it.

The server side applications, behavior applications 788, debrissimulations 789 and threat simulators 790 are supported on one or moreservers connected with the PBEG infrastructure, and these alsocommunicate with the PBEG infrastructure. In addition, a PBEG managerapplication 791 is connected with the PBEG infrastructure, and providesfor management of the PBEG infrastructure. The PBEG manager applicationsembrace a number of modules performing system wide functions to managethe distributed physics-based environment, such as load balancing,exercise control, failure recovery object or synchronizer state, etc.

The various applications are supported on a number of distributed clientcomputer systems, each of which supports a part of the PBEGinfrastructure, and are connected via a network, i.e., the physicscommunications backbone network. Generally, the PBEG infrastructurecomprises, at least, at each client computer system associated with it,a memory resident data storage that stores data representing theversion, local to that computer system, of a physics-based definition ofthe shared virtual environment that is updated as described herein.

FIG. 36 shows diagrammatically the general components of the PBEGinfrastructure in a distributed system with one SAF client computersystem 831, and two simulator stations 833 and 835. Client system 831includes SAF 832 software controlling operation of a SAF client entityin the virtual environment shared by the distributed systems. Thecomputer system 831 has a PBEG infrastructure 837 that stores in rapidaccess memory data constituting an object state list, which is made upof data records, one corresponding to each virtual object in the virtualenvironment, organized for rapid access by an appropriate data structureknown in the art, such as hash addressing. Each data record includesdata identifying the virtual object by a unique handle, identifiernumber or label and data defining its location, orientation, anddimensions, this being the data necessary to define the location andcondition of the object for generating its image. Color and texture ofthe object may be part of the data record as well, but are preferablystored elsewhere by the image generator, which accesses that informationwhen needed.

The client system further has a set of synchronizers that are softwareprograms running on the client system 837. One is a master synchronizer(“SAF Sync”) that transmits route data for the SAF client beingsimulated. Also included are synchronizers SIM2 Sync and SIM1 Sync,which are slave synchronizers that receive and implement in the objectstate data position/prediction data from the simulator stations. Theposition/prediction data is data indicating movement of the ownships ofthe vehicles simulated by the simulation stations 833 and 835. The datais derived from the movements of the ownships as defined by mastersynchronizers SIM1 Sync in station 833 and SIM2 Sync in station 835. Themovement of the ownship is generally controlled by the trainee at thesimulator and supportive software, and the data reflects thisman-in-loop input as well as any physical consequences on the associatedownship in the PBE.

The simulator stations each have respective controlling software 834 and836 that runs the simulator station to provide realistic interactionwith the human user, and also to receive input from the human user, asfrom simulated cockpit controls. The simulator stations each also have arespective viewer or image generator (IG) 841 and 842 that renders aview of the three-dimensional world defined by the respective storedobject state data 838 and 839 of the PBEG infrastructure. The viewersare rendering programs that render images as serial frames ofappropriate views from the point of view in the PBE of the respectivesimulator station, which frames are serially displayed as video at arate of approximately 60 Hz to provide realistic imagery in thesimulator.

In the preferred embodiment, the viewer renders its imagery based onscene data stored on a computer-accessible data storage device, whichscene data contains data for each object that defines its location,orientation, appearance, e.g., color, texture, etc. and any other dataneeded to generate an image of the object. The scene data is keptabsolutely up to date by constantly, i.e., at least once every framecycle (60 Hz in the preferred embodiment), querying the PBEGinfrastructure and deriving information regarding the location,orientation and other conditions of the objects therein, and updatingthe scene data accordingly. This information is stored in the objectstate list data, which generally is a list of every object in the PBEGenvironment or scene, with that physics data that is relevant to itsrepresentation in the image generator, meaning its location,orientation, and condition (e.g., exploding or dead), which will alsoaffect its appearance in the rendering. The viewer at the simulatorstation communicates directly with the PBEG infrastructure independentlyof the other simulator control applications 834 and 836, and can beconsidered, as illustrated in FIG. 35, as a separate applicationconnected with the PBEG infrastructure.

Referring to FIG. 37, the operation of the system is illustrated bylayers of functionality. The application level may include one of threepossible types of application that makes an object in the PBEGenvironment move: a SAF application that causes one or more vehicles orlife forms to move; a man-in-loop simulator, in which a human interactswith controls and causes the objects that make up the ownship of thesimulation to move in corresponding ways in the PBEG environmentresponsive thereto; and environment manager applications, which injectand manage physics based culture features (e.g., barrels, boxes, etc.)as well as activation of intelligent behaviors, such as the movement oflife forms or a crowd of civilians), and generally move objects based oninstructor input or any of a number of other automatic or scheduledevents that cause movement of objects in the PBEG environment.

These applications each interact with the PBEG infrastructure using arespective master controller. There are three types of mastersynchronizers: SAF controllers, man-in-loop controllers, andenvironmental controllers. SAF controllers transmit route data to slaveSAF controllers across the physics communications backbone network, ashas been discussed previously. Man-in-loop controllers transmit forcedata to slave man-in-loop controllers across the physics communicationsbackbone network, which translates the simulator application output tocause the ownship client objects in the various PBEG environmentversions to move as directed by the simulator application. Theenvironment controllers transmit data to slave environment controllersover the physics backbone that represents forces and also collision dataand positional update data, which is derived and applied generally asdiscussed above with respect to the first embodiments of the invention,i.e., by moving objects and resolving conflicts locally, and updatingthe positions of the client objects, which are then moved locally bytheir slave controllers to the proper location, which is uniform overthe entire distributed system.

The synchronizers, both master and slave, are essentially the sameprogram module loaded as a plug-in. As illustrated in FIG. 38, theclient system 845 that is assigned to be the master synchronizer systemhas a physics manager 846. The physics manager 846 is a softwareapplication that manages the local version of the PBEG object statedata. The physics manager 846 communicates with synchronizer plug-inmanager 847. Plug-in manager 847, when a client object assigned to it isinstantiated, accesses an object type controller plug-in mapping file isaccessed, which contains data that causes the plug-in manager to loadthe correct type of synchronizer (SAF, man-in-loop, or environmental),which is a plug-in that fits any client object of the specified type.Once the correct type of synchronizer plug-in 848 is loaded, it accessesa physical object configuration file that contains data identifying aparametric data file and a PhysX object description file that providesthe synchronizer with physics-based data that causes the plug-in toproperly manage the physics operation of the object, such as thesteering and acceleration of a vehicle.

Each plug-in then begins acting as a master synchronizer, andcommunicates with the local physics engine, which in the illustratedembodiment comprises a physics API 850 that communicates with thesynchronizers and with a physics processor API plug-in 851 that linksthe physics API with the physics processor 852, i.e., the hardwarephysics engine of the client system 845.

The physics backbone of the PBEG infrastructure connects with anotherclient system 855, and when the objects are instantiated, the physicsmanager application 856 of that system activates controller (i.e., slavesynchronizer) plug-in manager 857, which loads an appropriate slavesynchronizer plug-in 858 based on the object type/controller plug inmapping file for the type of client object that is to be controlled. Theslave synchronizer plug-in then accesses the physical objectconfiguration file, and through that file the parametric data file andPhysX object description file, which provide the slave synchronizerplug-in with data that is used to configure the plug-in to control thetype of client object. The slave synchronizer plug-ins 858 communicatewith the local physics engine, which in the illustrated embodimentcomprises a physics API 860 that communicates with the synchronizers andwith a physics processor API plug-in 861 that links the physics API withthe physics processor 862, i.e., the hardware physics engine of theclient system 855.

Subsequent to the initial loading, the master synchronizer 848 continuesto transmit the data appropriate to the type of object controlled, e.g.,for a SAF, route data, for a man-in-loop object, force data, etc., whichis received by the slave synchronizer 858 and used via the local physicsengine, to modify the local PBEG environment data stored in the localmemory at the client system 855, in a manner described above.

Referring to FIG. 39, the context of a client system is shownschematically. The client system comprises at least one of a simulatorapplication 865, a viewer 866, an environment application 867, or a SAFapplication 868, optionally connected with a DIS network 869 linking theclient system with an external entity manager using military standardprotocols. The external entity manager sends and receives entity data toand from applications that are not physics-based applications, mostnotably DIS and HLA.

The SAF application is often a traditional military computer generatedforce modeling application that has been modified or enhanced asnecessary to allow it to define modeled forces that can be introducedinto the PBEG environment. These types of SAF applications also involvessome additional packages, preferably, these being a SAF Physics API 870,a SAF Physics Object Manager 871, a SAF API 872, a SAF manager 873 andan artificial intelligence engine 874 for modeling intelligent behaviorsof SAF entities, all of which cooperate to provide for a class ofcomputer generated models to be injected into and managed in the PBEGenvironment.

All of these modules preferably link directly to the PBEG infrastructureof the client system, indicated at 875, or indirectly to the PBEGinfrastructure via physics application API which interfaces to the PBEGinfrastructure 875. The PBEG infrastructure 875 part of each computersystem is similar in each system, server or simulator station in thedistributed simulation system. It should also be understood that not allof the illustrated applications are used in every system. A simulatorstation, for example, may have only the simulator application 865 andthe viewer 866 in addition to the PBEG infrastructure 875. On the otherhand, in a server system with no operator, for example, only the SAFapplication and SAF modules are used with the PBEG infrastructure 875.

As best seen in FIG. 40, the PBEG infrastructure part 875 of each systempreferably comprises a physics environment API 951. The physicsenvironment API 951 provides the application interface to the PBEGinfrastructure 875, allows the applications accessing the PBEGinfrastructure 875 therethrough to create, remove, apply forces andtorques to virtual physical objects, and query virtual physical objectsinstantiated in the PBEG infrastructure via a handle or pointer usableto access the record via the object state data manager. The physicsenvironment API 951 also provides for initializing the PBEG environmentand registering for simulation events and managing event callbacks.Events may be object events, such as creation of the object, deletion ofthe object, or change in object appearance, or warfare events such asfire event for a weapon or detonation events for explosive devices.

The physics environment API 951 also provides for creating an object inthe PBEG environment, responsive to which the physics environment API951 returns a handle or pointer linking to the object state data recordin the object state data list. In addition, it provides for directing anentity object in the PBEG environment, as by giving the object a goal orapplying a force to it. It also responds to queries by applications,returning data corresponding to the object state for the object queried.

The object state manager 953 is an executable software module that isresponsible for managing an object state data structure or list storedin resident computer accessible memory. The object state list has arecord for every virtual object in the virtual scene, and is updated bythe object state manager continuously, e.g., once every calculationcycle of the physics engine, or once every rendering cycle of anassociated image generator. This update is performed by receiving fromthe synchronizers or the physics manager data reflecting the purelyphysics-based data stored for use by the physics engines, and bycombining data derived from that physics-based data that is useful ingenerating an image of the object, such as position and orientation,with other non-physics simulation data, such as health or appearancechanges, and attributes set during creation of the object in question,such as the type of entity that it is. The result is a constantly up topdate list that contains an organized data structure or list accessibleby the viewer application or any other application, that includes acomplete object state data record for each object in the PBEGenvironment. The object state manager 953 also processes object andwarfare events, and generates synchronizer requests based onsynchronizer requests. The object state manager also modifies the PBEGenvironment data at the given station to apply, for example, forcesindicated by the SAF or man-in-loop application that move an object inthe virtual world, via the physics manager 955.

Each object in the PBEG environment has a unique identification numberor code, using which the object manager can be queried to return itsobject state data. The object state manager includes utilities thatallow for efficient access of the object state data record for anyvirtual object in the PBEG environment. This may include the use of ahash table that convert complex entity types to integer, or organizationof classes of objects to describe appearance thereof.

The object state manager 953 also provides for initializing the PBEGenvironment and the objects therein. Objects are created when the objectmanager receives a create object instruction from the physicsenvironment API, and remote objects are created when a create-objectevent is received from the PBEG Environment Event handler, which will bediscussed below. The object state manager 953 notifies subscribingapplications of created, removed, or objects changed in appearance as anobject state notification event.

The object state data is directly based on the current physics-basedenvironment data stored in the local computer-accessible memory, whichis the foundational data that gives rise to the displayed views of thePBEG virtual environment prepared by the image generator or viewer, andalso is the data accessed by the physics engines to ensure that physicallaws are followed in the simulated environment. Unlike other systemsthat rely on the rendering engine and its appearance-related scene dataas the foundation of the system that defines the appearance of thevirtual world, the present system relies on the physics-basedenvironment data, and then, based on the physics-based environment data,accesses data defining the appearance of the virtual world being viewedat any particular simulation station.

Physics manager 955 is preferably an executable software module thatmanages physics data and activities of the system, i.e., it manages thesystem components that create and interact with the physics basedenvironment. Physics manager 955 initializes the physics-based datastored for access by the physics engines by loading the entirephysics-based scene database inform a repository 957, i.e., acomputer-accessible mass-storage device accessible from the computersystem in which the physics manager runs. The PBEG repository containsdata that defines the layout of synchronizer objects, physics models,and attributes needed by the PBEG environment to load and manage physicsmodels. The repository provides a utilities package to access therepository.

In addition, the physics manager causes the PBEG synchronizer manager toload the synchronizers for the synchronizer system described above. Thephysics manager maps the modeled object type and synchronizer class(e.g., man-in-loop, SAF, etc.) to repository naming conventions so thatthe physics manager, when setting up synchronizers for an object, anddirects the synchronizer manager to load the appropriate synchronizerfor the given object. The physics manager also acts similarly to thephysics farm manager discussed above in that it will assign whichsynchronizer is to be the master synchronizer.

Once the synchronizers are loaded, the physics manager transfers to thelocal master synchronizers the movement or force data received from thecontrolling application, which implement it directly to the physicsengine. For example, the simulation application of a simulator stationtransmits or passes to the local physics manager a pilot's cockpitinputs, which are then passed on to a master synchronizer for theownship, or for an object that is a part of the ownship, of thesimulator station.

The physics manager or the object state manager also communicate theroute data for an object that is received over the network or physicsbackbone to the appropriate slave synchronizer responsible for theobject, which processes the route data as set out above.

For efficiency, the virtual world that is shared among the simulatorstations is preferably broken up into separate scenes. If a user movesfrom on scene to another, the physics manager will load a new scenedatabase to the local physics engine, and unload the former scenedatabase, as necessary.

Also, the physics manager receives warfare events from the object statemanager, and creates forces corresponding to them in the physicsenvironment stored for access by the physics engines. The physicsmanager uses stored warfare data, munitions types, and other dataregarding weapons in the simulation to determine the magnitude of anyimpulse force and inputs this to the physics-based environment data. Thescale of munitions impulse may be precise, or it may be simply classedas small, medium or large.

Synchronizer pluggins manager 958 receives requests to loadsynchronizers from the repository, and the proper type of synchronizerspecified by the physics manager is loaded. The synchronizer plugginsmanager 958 then schedules the loaded pluggin to run and load theappropriate physic module for the client object of that synchronizer.Once the physics model is loaded, the synchronizer pluggins manager 958is set up and runs in the shell of the pluggins manager 958. Thesynchronizer pluggins manager 958 schedules the synchronizer to run, andunloads the synchronizer and associated physics models responsive toreceiving or detecting that there is a request to remove the object.

The three general types of synchronizer pluggins discussed above areshown in the diagram of FIG. 40, but there are normally a very largenumber of synchronizers, both master and slave, managed by thesynchronizer pluggins manager 958 in each system, since there is onesynchronizer per object.

These synchronizers operate as described above in detail, and theycommunicate with the physics engines 959, which are a wrapper around theCOTS physics engines such as described previously, such as the PhysXengine sold by Ageia. The wrapper supports plug and play, and can beused with a variety of different COTS physics engines 959. Thesynchronizers provide force or torque data to the physics engines, whichthen calculate a solution that reflects the physical consequences,pursuant to the physical laws of the physics engines, of the forces ortorques introduced by the synchronizers on the objects defined by thephysics data. The physics engines calculate this solution periodically,preferably at least once per image rendering cycle (preferably about 60Hz), and the result is modifications to the physics data stored in thephysics engine memory reflecting movement of the defined virtualobjects.

The synchronizers and the physics engines exchange data back and forthto accomplish movement of the objects as directed by the associatedapplication, and in compliance with the physical laws of the physicsengines 959. The interaction preferably includes a closed-loop controlof the parameters of the client object of the synchronizer, as wasdiscussed above. In addition, the synchronizers, whether master orslave, transmit at least some of the physics data back to the objectstate manager, which takes the position, orientation, and possiblyvelocity or other physical qualities of use in determining theappearance of the object, and stores that viewer-relevant data in theobject state data structure or list. As a result, the object state listis an immediate reflection of the location of all the objects defined bythe physics data defining the virtual scene as stored in the physicsengine memory. In turn, the scene data used by the image generator orviewer to render the imagery displayed reflects the current state of theobject state data. The result is that the viewer displays to thesimulator user imagery that reflects the virtual environment exactly asit is defined at the moment by the local physics data in the physicsengine memory.

PBEG Event Handler 961 provides an interface for the PBEG environment togenerate and distribute event messages. It provides for creation anddistribution of war events, e.g., fire, or detonations. The eventhandler 961 also creates object notification events, such as creation,removal, and change of appearance of an object. It also createssimulation management events, and provides a way for components of thesystem to register for event callbacks, object state, simulationcontrol, and other event notices. Also, the event handler managesregistered callbacks, and distributes events based thereon.

The event handler 961 communicates the events, which are the realtransactional elements of the system, to the local physics manager 955and to the network link. Events are any occurrence within the system forwhich it is helpful or advisable to alert other client systems to, suchas the creation of an object, or its transformation, as by explosion,into something else. The event handler also receives the broadcast routedata from all the master synchronizers in the system and transmits it tothe object state manager 953, which communicates the route data to theappropriate slave synchronizers or controllers for processing into thephysics engine as has been described above.

Physics environment control API 962 provides the functionality thatallows an instructor to design, implement and/or edit a trainingexercise by registering client applications, setting the environment,starting or terminating the exercise, or removing the entity. It issimilar to, or even redundant with, the exercise management modulediscussed above.

Events and synchronizer data transmissions go to other client systemsover the physics communication backbone 971 to which the client isconnected by link layers. Low-level network layer 972 is responsible foropening connections, closing connections, sending data, and receivingunreliable data. Reliable data is processed at the application levelnetwork 973.

The network support is based on UDP, the standard Internet protocol forunreliable, connectionless communication. Because of this, the networklayers are needed, to be responsible for maintaining a list ofconnections and handling their time-outs. The network layer ispacket-oriented, meaning that all data sent and received via the networklater exists in discrete packets.

The network will send the packets of data unreliably. A packet sent fromone client system to another might not arrive, might arrive once, ormight arrive multiple times. The packets might also arrive out of orderneeding reassembly. The network layer takes steps to ensure that packetsare not corrupted. These include checking the packet that its size andcontents are identical to what was sent.

Preferably, the network layer is portable and multi-thread capable. Itconnects all of the client systems and servers, and any distributed PBEGmanagement applications are routed therethrough. Its essential functionis to optimally send PBEG messages on the PBEG communications backbone971 these services employ multicast schemes to partition traffic, as byexercise, by scene, to pack and unpack messages, and other communicationservices to optimize the overall data sharing and communication of thenetwork of the backbone.

The application-level network 973 layer sits on top of the low-levelnetwork and provides services such as reliable message handling,multicast, message-compression, network statistics, and other messagepackaging optimizations.

FIG. 41 illustrates the flow of commands in certain types of systemactions, in this case, initialization of the system. The application,the simulator application or an administrative startup for a clientsystem, starts a routine named PBEG Initialize, which transmits arequest to the physics environment API to initialize. The physicsenvironment API transmits the instruction to the object state manager,which tells the physics manager to initialize the station. The physicsmanager initializes the synchronizer manager so that it loads theappropriate synchronizers, and the event handler, which sends out anotification over the network. The physics manager also initializes thephysics engine, which normally involves loading the physics-based scenedata into the resident memory associated with the physics engine orengines in the client system.

FIG. 42 illustrates the creating of a new vehicle in the system of theinvention.

The application, which has adequate administrative capabilities tocreate an object, starts the CreateObject routine, which communicateswith the physics environment API, which in turn communicates with theobject state manager to create the object. The object state managercauses the physics manager to load the object from the synchronizerphysics model repository connected with the client system on which theprograms are running. The physics manager determines the appropriatetype of synchronizer, and publishes the notice of a created object toall the other systems over the network, here illustrated as the remote,below the black line. In addition, the physics manager directs thesynchronizer manager to load the proper synchronizer plug-in andinitialize. The synchronizer for the new object initializes, creating anew master synchronizer for the object, and loads the physics model fromthe repository into the physics engine memory, and creates the newobject in the physics engine, an action which provides in response, ahandle to the physics object in the PBEG environment data that thephysics engine accesses. This is returned through the physics manager tothe object state manager, which creates a record for the new object'sobject state, and transmits a handle for that object state data recordback to the physics environment, which is passed back to theapplication, or another local application as well, such as the viewer atthe local simulation station.

The published create-object instruction goes through the physicsbackbone to the remote system, where it is received by the eventhandler, which directs the local object state manager to create theobject. In reaction, the local object manager issues a load objectinstruction to its physics manager, which in turn tells its synchronizermanager to load the plug-in and initialize. The synchronizer loads thesynchronizer, which initializes as a slave synchronizer, and a physicsmodel from its repository, and creates the object in the PBEGenvironment physics data, with routine CreateActor, accessing thephysics engine and the version of the PBEG environment data that isstored at the remote system. The physics engine returns a handle(meaning a unique pointer, identifier or address) to the created physicsobject in the physic engine resident memory, which is passed back to theremote object state manager, which places a record in the object statedata at the remote, and transmits a handle linking to that object statedata in the object state database. The two client systems, local andremote, after this process, will display the new object insynchronization, provided the object is in the virtual field of view ofthe respective viewers.

The relationship of these components on a given server or client systemis illustrated in FIG. 43. The application 980, which may be any type ofobject-controlling application, e.g., a simulator, a SAF model, anenvironmental model, etc., interacts with the surface interfaceapplication of the PBEG infrastructure, physics application API 981,transmitting to it data indicative of movement or change of the clientobject affected by application 980. Physics application API 981 receivesthat information and transmits or passes instructions create, delete orupdate the object state manager 982. The object state manager 982receives this data and transmits to the physics manager 983 datacontaining the instructions to create or delete the object, or to thesynchronizer 984 data containing instructions to update the physicsobject. If the instruction is to create the object, the physics manager983 loads a controller or synchronizer plug-in from a repository 985,and this becomes the synchronizer 984, which loads into the residentdata memory of the physics engine 987 a physics model for the objectfrom a stored repository 986, which may be the same device as therepository 985. If the instruction is to update the object in terms offorces or position, that data is also routed through the synchronizer bya closed loop control to the physics engine memory. Physics manager 983only accesses the memory of the physics engine if the instruction is toinitialize the physics engine 987, in which case, a full scene ofphysics data is loaded into the physics engine 987 memory. In the eventthat the update to the object is a transformative event like anexplosion, the object state manager 982 passes the instruction to theevent handler 988, and also to the synchronizer to transmit theappropriate physics data of the transformation.

Data also flows in the reverse direction. There is the reverse flow ofdata to the synchronizer 984 from the physics engine 987 as part of theclosed loop controller process thereof. This data is returned from thephysics manager 983 or the synchronizer 984 to the object state manager982, together with a handle for accessing the physics of the object if aquery is to be made to the physics engine memory. This data is combinedwith other information to yield the object state data record in theobject state list, in which the object's data record includes datastoring the handle identifying it. In turn, a handle or addressidentifying the object's object state data record may be returned to theapplication via the interface of physics application API 981 to thephysics application 980.

As best seen in FIGS. 44 and 45, a PBEG exercise management module 990supported on a separate remote server, provides for exercises fortraining in the PBEG environment. The exercise management module 990provides a variety of functions, including selecting the virtual worlddesign for the physics based PBEG environment, e.g., the Arizona desert,from a library, and also setting up the PBEG or controlling a trainingexercise. The PBEG exercise management allows for applications to jointhe system and participate in an exercise even after it has started,and, also, applications can re-join an exercise after failure.Applications register themselves with the PBEG exercise managementcomponent and respond to PBEG exercise management controls. Preferably,provisions are also made for applications to operate in a stand-alonemode, without the PBEG exercise management.

The control flow for the start of an exercise is illustrated in FIG. 44.The PBEG exercise management module 990 transmits a message indicatingexercise start across the physics backbone to the event handler at theclient system for the application to be involved. The event handler 991routes this notification to the physics environment control API 992 onthe outer functionality layer of the PBEG infrastructure, whichtransfers the notification to the application 993 itself as an exercisecallback message requesting a call back to the PBEG exercise managementmodule 990. A callback is issued, either automatically or by operatorinput, acknowledging the start notification, which is passed into thephysics infrastructure at API 992, then to the event handler 991 andover the network back to PBEG exercise management module 990.

A request for callback may also be made in the other direction asillustrated in FIG. 45. The application 993 transmits a registrationrequest to the exercise management module 990. it enters the PBEGinfrastructure through API 992, is passed to the event handler 992,which transmits the registration message over the network layer andphysics backbone to the PBEG exercise management module 990. The PBEGexercise management module 990 then responds with an acknowledgementmessage that returns through the exact same route to application 993,arriving as an acknowledge callback.

The foregoing invention provides significant improvement over earlierdistributed systems. The object state manager, which organizes objectstate data allows for a more efficient aggregation of virtual objects,even when they are the products of applications as different as legacySAF systems and current simulator stations. The system and methodsherein avoid latency as well as avoiding the problems of tight couplingbetween the behavior, physics and rendering, which has presentedproblems in the prior art. The use of plug ins as synchronizers and forthe physics engines provides a very flexible open architecture, and thesystem of the invention is highly scalable.

It will be understood that the systems and methods of the foregoingdisclosure are preferably implemented using software operating onelectronic computer processor systems linked via a network such as theinternet and having computer-accessible data storage attached theretosuch as disk drives or other mass storage devices. The physics enginecomponents, however, as discussed above, are preferably dedicatedhardware implemented in a computer system having memory and displaydevices as are well known in the art.

As has been mentioned above, under certain circumstances, the mastersynchronizer or controller for an object may be replaced by a differentmaster synchronizer at a different client system on the network. Theprocess by which this happens is illustrated in FIG. 46

When the object is first instantiated by application A in FIG. 46, acreate-entity instruction 991 is sent through the physics environmentAPI at computer system 992 to create master Synchronizer A 993. Mastersynchronizer A 993 sends a command over the physics network to loadappropriate slave synchronizer A, which is received at a second computersystem 994, which loads and starts slave synchronizer A 995.

At station 994 an application B is started, which may be a manualcontrol or another type of controlling software for the entity that hasbeen created. Application B triggers loading of master synchronizer B996 by an add synchronizer command 997. This command may be the resultcaused by Application B when an instructor at station 994 activatesdirect human control of an entity at the instructor station, and themaster synchronizer program starts on the instructor station computer.Loading of master synchronizer B produces a command to load slavesynchronizer B, which is broadcast over the entire system, where everysystem, including system 992, loads a slave synchronizer B 998.

Subsequently, Application B is prompted by the user or by some otherevent to activate control by Application B. this results in anactivate-synchronizer command 999 sent to the physics environment API atsystem 994, which does two things: it de-activates the then-active slavesynchronizer following master synchronizer A, and it activates mastersynchronizer B to take control of the entity.

When de-activated, slave synchronizer A 995 sends a deactivationinstruction to its master synchronizer A 993 over the network. Themaster controller 993 at the simulation station 992 controlling theentity then detects the presence of the superseding master synchronizerand relinquishes control, no longer transmitting any route data.

When activated, the new master synchronizer B 996 transmits aninstruction to activate all the corresponding slave controllersthroughout the system, including slave synchronizer B on client system992. All the slave synchronizers loaded for master synchronizer B thenbegin to control their respective local copies of the entity accordingto the control data received from master synchronizer B 996, as operatedby Application B. The control data may be route data or man-in-loopcontrol data, depending on what type of application Application B is.

The process can be reversed responsive to Application A assertingcontrol again with an activate synchronizer instruction to system 992,or responsive to a deactivate synchronizer command from Application B atcomputer system 994.

The terms used in this specification should be understood to be languageof description, not limitation, as those of skill in the art with thisspecification before them will be able to make changes and modificationstherein without departing from the scope of the invention. Inparticular, the term SAF as it has been used herein should not be seenas limited to legacy types of object force simulation, but as referringto any sort of moving entity or object that exhibits acomputer-generated behavior, especially, but not limited to,computer-controlled opposing forces that act according to predefinedtactical rules. Similarly, the terms object and entity have been usedfairly interchangeably herein, both intended to refer to virtual objectsor virtual collections of object defined by data stored in electroniccomputer systems.

What is claimed is:
 1. A simulation system comprising: a plurality ofcomputerized simulation stations connected with each other via anetwork, each having a respective computer-accessible data storagestoring a respective version of scene data corresponding to a virtualenvironment, a respective computerized image generator generating inreal time serial frames each of which is a rendered view for arespective point of view in said virtual environment, and a displaydevice displaying said frames to a respective user of the computerizedsimulation station; said versions of the scene data each includingrespective client data defining a virtual client entity in the virtualenvironment, said client data including respective position dataassociated with the virtual client entity and defining a respectiveposition thereof in the respective version of the virtual environment;said computerized simulation stations each modifying the respectiveclient data in real time pursuant to the respective version of the scenedata and to communications over the network such that the respectiveversion of the scene data defines the virtual client entity moving inthe virtual environment according to predetermined physics rules definedin the computerized simulation station controlling movement of virtualobjects in the virtual environment; wherein each simulation station hasa physics engine accessing physics based environment data stored at saidsimulation station corresponding to all virtual objects in the virtualenvironment of the scene data, the predetermined physics rules beingimplemented by said physics engine, said physics engine controlling therespective version of the scene data such that the virtual client entityand the virtual objects in the respective version of scene data moveaccording to the physics based environment data and the physics rules;wherein the simulation stations each transmit physics based data overthe network corresponding to physical forces applied to some of thevirtual objects in the virtual environment; a simulation computer systemdetermining behavior of the virtual client entity in the virtualenvironment and transmitting client synchronization data via the networkto the simulation stations, said client synchronization datacorresponding to said behavior and defining a location of the cliententity in the virtual environment, and time-constraint data indicativeof a point in time for the entity to be at the location; and eachsimulation station having a slave synchronizer receiving the clientsynchronization data and based thereon causing the respective version ofthe scene data to be modified so as to reflect movement towardsynchronization of the location of the client entity in the respectiveversion of the scene data with the virtual client entity being at saidlocation at said point in time according to the predetermined physicsrules and the respective version of the scene data so as to reduce anylatency-caused differences between the positions of the virtual cliententity in the versions of the scene data.
 2. The system of claim 1,wherein the client synchronization data identifies the location as apoint in the virtual environment.
 3. The system of claim 2, wherein,where the client entity is a ground-moving entity, the clientsynchronization data identifies the location by a two-dimensionalcoordinate in a two-dimensional surface in the virtual environment inwhich the client entity moves.
 4. The system of claim 2, wherein, wherethe client entity is an air-moving entity, the client synchronizationdata identifies the location by a three-dimensional coordinate in athree-dimensional space in the virtual environment in which the cliententity moves.
 5. The system of claim 1, wherein the clientsynchronization data identifies an orientation in the virtualenvironment for the client entity.
 6. The system of claim 1, wherein theclient synchronization data comprises data defining a point in time atwhich the entity is directed to reach the location.
 7. The system ofclaim 1, wherein the client synchronization data comprises data defininga rate at which the entity is directed to move toward the location. 8.The system of claim 7, wherein the client synchronization data comprisesdata defining a start time at which the entity is directed to startmoving toward the location.
 9. The system of claim 1, wherein thesynchronizer controls the movement of the client entity in the scenedata using a closed loop control process.
 10. The system of claim 1,wherein each synchronizer influences the respective version of the scenedata by transmitting data to the physics engine configured to producemovement of the client entity in the virtual environment.
 11. The systemof claim 1, wherein positional update data is transmitted to thesimulation stations so that the scene data is updated continually so asto be in conformance with all other versions of the scene data acrossthe system.
 12. The system of claim 1, wherein the client entity isdirected to move along a route to said location, and wherein, if theclient entity is prevented from movement toward the location by thephysics engine due to obstruction of said route by an object or objectsin the virtual environment, a new route to the location is devised thatavoids being obstructed by said object or objects, and said cliententity is caused by said synchronizer to move along said new route so asto reach the location by said point in time.
 13. The system of claim 1,wherein the client entity is directed to move along a route to saidlocation, and wherein, if the client entity is prevented from movementtoward the location by the physics engine due to obstruction of saidroute by an object or objects in the virtual environment, saidsynchronizer causes the physics engine to temporarily suspend at leastpart of the physics rules thereof with respect to the object or objectssuch that the client entity is able to move without being obstructedthereby and so as to reach the location by said point in time.
 14. Thesystem of claim 1, wherein, responsive to the client movement data, thesynchronizer is directed to move along a first route to said location,and, if the client entity is prevented from movement toward the locationby the physics engine due to obstruction of said route by an object orobjects in the virtual environment, the synchronizer attempts to devisea new route to the location that avoids being obstructed by said objector objects, wherein, if the synchronizer can devise the new route, saidclient entity is caused by said synchronizer to move along said newroute so as to reach the location by said point in time; and wherein, ifthe synchronizer can devise the new route, said synchronizer causes thephysics engine to temporarily suspend at least part of the physics rulesthereof with respect to the object or objects such that the cliententity is able to move along the route without being obstructed therebyand so as to reach the location by said point in time.
 15. The system ofclaim 1, wherein the synchronizer derives from the client movement dataa desired route for the client entity to follow to the location and adesired rate at which to proceed along said route, the synchronizerusing a closed loop speed control process to control the rate ofmovement of the client entity along said route based on the desiredrate, and the synchronizer using a closed loop steering control processto control the direction of movement of the client entity so that saidclient entity follows said route.
 16. The system according to claim 1,wherein the simulation stations each emulate a vehicle and include inputdevices that allow input of control commands from the respective user,said control commands being used by the respective simulation system toinfluence the version of scene data.
 17. The system according to claim16, wherein the simulation stations each transmit client synchronizationdata for a respective client vehicle entity, and the other simulationstations each has a synchronizer that moves the client vehicle entity inthe respective version of the scene data according thereto and accordingto the respective physics rules.
 18. The system according to claim 1,wherein the synchronizer compares a position of the virtual cliententity derives difference data from a comparison of a synchronizedposition derived from the client synchronization data with a currentposition of the virtual client entity, and deriving physics data causingthe virtual client entity to move toward the synchronized position. 19.The system according to claim 1, wherein the simulation computer systemhas a simulated vehicle receiving input from a trainee and a mastersynchronizer, said master synchronizer deriving the clientsynchronization data.
 20. The system of claim 1, wherein the simulationcomputer system causes the virtual client entity to simulate operationof a vehicle responsive to input from a human operator interacting withthe simulation computer system.
 21. The system of claim 1, wherein thesimulation computer system determines the behavior of the virtual cliententity as an automated force acting solely based on automatic computercontrol thereof.
 22. A system for providing distributed simulations toone or more users, said system comprising: a first computer systemlinked for communication over a network and having a data storage devicestoring data and a display; said data storage device storing scene datadefining a first version of a virtual scene having a number of virtualobjects therein each having a respective visual appearance, position andorientation defined by said scene data; said computer system including aviewer generating real time imagery from said scene data and displayingsaid imagery on said display, the imagery corresponding to a renderedview at the time the image was rendered of the virtual scene as definedby said scene data; a second computer system having a data storagedevice, said data storage device storing scene data defining a secondversion of the same virtual scene with the same virtual objects asdefined by said data on the first computer system; said first and secondcomputer systems each independently determining the positions of virtualobjects in the virtual scene based on the scene data defining therespective version of the virtual scene, predetermined physics rulesdefining interactions between the virtual objects, and communicationsreceived over the network relating to some of the virtual objects;wherein each computer system has a physics engine with memory therein,said physics engine setting the predetermined rules defininginteractions between the virtual objects and storing physics datacorresponding to physical parameters of the virtual objects in thevirtual scene including respective positions thereof, said scene data ateach computer system being periodically updated by position data fromthe physics engine thereof; wherein the computer systems transmitphysics data between the physics engines over the network, said physicsdata including physics data units each being determined at a respectiveone of said physics engines and corresponding to a respective virtualforce applied to a respective one of the virtual objects, wherein thevirtual force is applied to said virtual object by the other of thephysics engine; said second computer system running a programdetermining movement of a controlled virtual object in the virtualscene; said second computer system further having a master synchronizertransmitting over the network route data for said controlled virtualobject indicative of a point in the virtual scene and a future point intime by which the controlled virtual object is directed to reach saidpoint; said first computer system having a slave synchronizer receivingsaid route data, and responsive thereto, causing the scene data storedtherein to be modified so that in the scene data stored at the firstcomputer system, the controlled virtual object moves to the point in thevirtual scene by the point in time in conformance with the predeterminedphysics rules defining interactions between the virtual objects so as toreduce any latency-caused difference between the positions of thecontrolled virtual object as determined by the first and second computersystems.
 23. The system of claim 22 wherein the point in time is definedin terms of a shared system clock value.
 24. The system of claim 22wherein the route data comprises data defining a point and data defininga point in time.
 25. The system of claim 22 wherein the route datacomprises data defining a point in the scene, a start time and a rate.26. The system of claim 22 wherein the first computer system has amaster synchronizer that begins to transmit route data for the virtualobject to the second computer system, and said second computer systemhas a slave synchronizer that modifies the data stored at the secondcomputer system so as to comply with said route data, and the mastersynchronizer at the second computer system relinquishes control of thecontrolled virtual object and the slave synchronizer at the firstcomputer system does not cause further changes in the data at the firstcomputer system.
 27. The system of claim 22 wherein the slavesynchronizer at the first computer system determines whether the virtualobject is delayed in proceeding to said point by said time, andresponsive to said determination, takes a corrective action selectedfrom the group consisting of increasing force on the virtual object,causing the virtual object to move on a route around an obstruction, andmodifying or suspending physical rules of the physics engine so as topermit the virtual object to reach the point by the time.
 28. The systemof claim 22, wherein the second computer system determines the movementof the controlled virtual object as an automated force with a behaviordetermined by the second computer system.
 29. A system for providingsimulations to one or more users, said system comprising: a plurality ofcomputerized simulation stations each linked with and communicating overa network so as to transmit data to each other thereacross; eachsimulation station having a data storage device storing scene datadefining a respective version of a shared virtual environment includingrespective positions of virtual entities in said version of said virtualenvironment; each simulation station determining and transmitting overthe network physical data indicative of virtual force being applied inthe simulation to an associated one of the virtual entities, and eachsimulation station receiving over the network from one of the othersimulation stations physical data defining respective virtual forcesbeing applied in simulation for one of the other virtual entities; eachof said simulation stations including a physics engine determining inreal time positions of the virtual entities in the respective version ofthe scene data according to predetermined physics rules based on thephysical data, on physics data in the scene data defining physicalattributes in simulation of the virtual entities and virtual objects inthe virtual environment; an image generator generating video for thesimulation in real time from the respective scene data; and a displayapparatus displaying the video to the respective user in real time; eachsimulation station transmitting to all other simulation stationssynchronization data for the associated virtual entity defining aposition in the virtual environment and a point in time for theassociated virtual entity to be at said point; each simulation stationreceiving over the network synchronization data for a virtual entityassociated with another simulation station, and having a slavesynchronizer determining synchronizer forces that are applied to saidvirtual entity by the physics engine thereof so as to cause said virtualentity to move toward a position based on said received synchronizationdata in conformance with the predetermined physics rules so as to reduceany latency-caused differences between the positions of the virtualentity in the versions of the scene data.
 30. The system according toclaim 29, wherein the synchronizer forces are determined by a controllerthat compares a current position of the virtual entity with a desiredposition thereof and calculates forces configured to move said virtualentity toward the desired position in the version of the scene data. 31.The system according to claim 30, wherein the controller operates in acontrol loop so as to continually determine said forces to move thevirtual entity to the desired position.
 32. The system according toclaim 30, wherein, responsive to a determination that the virtual entityis delayed in proceeding to said desired position by said point in time,the simulation station takes a corrective action selected from the groupconsisting of increasing force on the virtual entity, causing thevirtual entity to move on a route around an obstruction, and modifyingor suspending the physics rules of the physics engine so as to permitthe virtual entity to reach the desired position by the point in time.33. The system of claim 29, wherein the simulation stations each includea simulator of a respective vehicle, and the associated virtual entitycorresponds to the respective vehicle in the virtual environment, saidsimulation system determining movement of the associated virtual entityresponsive to user input from the simulator.
 34. A system for providingsimulation of a shared virtual environment to a plurality of users, saidsystem comprising: a network connecting a plurality of computerizedsimulation stations each associated with a respective user and storing arespective version of simulation environment data defining the virtualenvironment; and said simulation stations exchanging over the networkphysics force data defining forces applied to virtual objects in thevirtual environment and synchronization data defining mastersynchronizer time and location data in the virtual environment forcorresponding virtual objects determined by an assigned respective oneof the simulation stations; each simulation station having a slavesynchronizer determining synchronizer physics force data from thesynchronization data for the associated virtual object not assigned tosaid simulation station, said synchronizer physics force data beingconfigured to move the virtual object toward a position based on themaster synchronizer time and location data of the synchronization dataso as to reduce any latency-caused differences between the positions ofthe virtual object in the versions of the simulation environment data;each simulation station determining respective positions in the versionof the simulation environment data thereof for virtual objects based onthe physics force data and predetermined physics rules so that positionsof the virtual objects are synchronized across the system in conformancewith the physics rules; and wherein each simulation station determinessaid respective positions in the version of the simulation environmentdata thereof for the virtual objects using a physics processing unit ofthe simulation station applying the physics force data.
 35. The systemaccording to claim 34, wherein the simulation environment data includesdata defining the position, orientation, mass, dimensions, andappearance of all virtual objects in the virtual environment.
 36. Thesystem according to claim 35, wherein for each of the virtual objectsthe master synchronizer time and location data thereof is generated byonly one of the simulations stations.
 37. The system according to claim36, wherein the simulation station generating the master synchronizertime and location data operates as a master synchronizer, and all othersimulation stations process the master synchronizer time and locationdata as slave synchronizers.
 38. The system according to claim 34,wherein the simulator stations each has an image generator renderingvideo in real time of the virtual environment based on the respectiveversion of the simulated environment data, and a display devicedisplaying said video to the user.
 39. The system according to claim 34,wherein the simulation stations each determine the synchronizer physicsforce data based on a comparison of the position of the virtual objectaccording to the respective version of the simulation environment datawith a desired position of the virtual object based on the mastersynchronizer time and location data of the synchronization data.
 40. Thesystem of claim 34 wherein the simulation stations each include asimulator of a respective vehicle, and the associated virtual entitycorresponds to the respective vehicle in the virtual environment, saidsimulation system determining movement of the associated virtual entityresponsive to user input from the simulator.